[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16936909#comment-16936909 ] Julia Kinga Marton commented on OOZIE-3542: --- [~zsombor], can you please share what was the problem with the initial patch? > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3533) Flaky test TestXLogService.testLog4jReload
[
https://issues.apache.org/jira/browse/OOZIE-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16935681#comment-16935681
]
Julia Kinga Marton commented on OOZIE-3533:
---
[~asalamon74] Option 2 is the most sympathetic to me.
Since only sometimes the 1000 ms is not enough, I don't think that we should
force a bigger sleep value for every case.
> Flaky test TestXLogService.testLog4jReload
> --
>
> Key: OOZIE-3533
> URL: https://issues.apache.org/jira/browse/OOZIE-3533
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
>
> Sometimes this test fails with the following error message:
> {noformat}
> junit.framework.AssertionFailedError
> at junit.framework.Assert.fail(Assert.java:55)
> at junit.framework.Assert.assertTrue(Assert.java:22)
> at junit.framework.Assert.assertFalse(Assert.java:39)
> at junit.framework.Assert.assertFalse(Assert.java:47)
> at junit.framework.TestCase.assertFalse(TestCase.java:219)
> at
> org.apache.oozie.service.TestXLogService.testLog4jReload(TestXLogService.java:111)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at junit.framework.TestCase.runBare(TestCase.java:141)
> at junit.framework.TestResult$1.protect(TestResult.java:122)
> at junit.framework.TestResult.runProtected(TestResult.java:142)
> at junit.framework.TestResult.run(TestResult.java:125)
> at junit.framework.TestCase.run(TestCase.java:129)
> at junit.framework.TestSuite.runTest(TestSuite.java:255)
> at junit.framework.TestSuite.run(TestSuite.java:250)
> at
> org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at org.junit.runners.Suite.runChild(Suite.java:127)
> at org.junit.runners.Suite.runChild(Suite.java:26)
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:410)
> at
> org.apache.maven.surefire.junitcore.pc.InvokerStrategy.schedule(InvokerStrategy.java:54)
> at
> org.apache.maven.surefire.junitcore.pc.Scheduler.schedule(Scheduler.java:367)
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junitcore.pc.ParallelComputerBuilder$PC$1.run(ParallelComputerBuilder.java:593)
> at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:373)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:334)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:119)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:407)
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928278#comment-16928278 ] Julia Kinga Marton commented on OOZIE-3542: --- Thank you [~zsombor] and [~dionusos] for the fix! +1, committed to master > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[
https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927534#comment-16927534
]
Julia Kinga Marton commented on OOZIE-3542:
---
[~dionusos], I yesterday [~zsombor] mentioned offline that the
{{SystemErasureCodingPolicies}} is called using reflection, so is not a good
idea to rename it. I think that the test failures are caused by this rename.
Sorry for the confusion.
> Handle better old Hdfs implementations in ECPolicyDisabler
> --
>
> Key: OOZIE-3542
> URL: https://issues.apache.org/jira/browse/OOZIE-3542
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Major
> Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch
>
>
> Currently, ECPolicyDisabler checks if the local hdfs implementation has the
> necessary methods to get and set erasure coding policy. However, if the
> namenode implementation is old, it could throw a
> org.apache.hadoop.ipc.RemoteException with
> RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it.
> In this case, ECPolicyDisabler fails, and prevents the installation to
> succeed.
> This case should be handled just like, when erasure coding is not supported.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927333#comment-16927333 ] Julia Kinga Marton commented on OOZIE-3542: --- Thanks [~dionusos]! LGTM, +1, let's wait for the pending Jenkins and I will merge the change. > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926474#comment-16926474 ] Julia Kinga Marton edited comment on OOZIE-3542 at 9/10/19 9:40 AM: Thank you [~zsombor] for the fix. I would have two small comments related your patch: * Please rename the test class to TestECPolicyDisabler.java * Please mark somewhere in the name of SystemErasureCodingPolicies that is created for tested purposes, so when we are doing class search to know it even without opening it. (for example I would rename it to SystemErasureCodingPoliciesForTest) was (Author: kmarton): Thank you [~zsombor] for the fix. I would have two small comments related this: * Please rename the test class to TestECPolicyDisabler.java * Please mark somewhere in the name of SystemErasureCodingPolicies that is created for tested purposes, so when we are doing class search to know it even without opening it. (for example I would rename it to SystemErasureCodingPoliciesForTest) > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926474#comment-16926474 ] Julia Kinga Marton commented on OOZIE-3542: --- Thank you [~zsombor] for the fix. I would have two small comments related this: * Please rename the test class to TestECPolicyDisabler.java * Please mark somewhere in the name of SystemErasureCodingPolicies that is created for tested purposes, so when we are doing class search to know it even without opening it. (for example I would rename it to SystemErasureCodingPoliciesForTest) > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Attachments: OOZIE-3542-2.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924366#comment-16924366
]
Julia Kinga Marton edited comment on OOZIE-3405 at 9/6/19 4:16 PM:
---
[~matijhs], I can see in the review that you have uploaded multiple patches.
Can you please upload your patch here as well, to trigger the precommit script,
or this one is the last one?
was (Author: kmarton):
[~matijhs], I can see in the review that you have uploaded multiple patches.
Can you please upload your patch here as well, to trigger the precommit script?
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.1.0
>Reporter: Peter Orova
>Assignee: Mate Juhasz
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-3405-V1.patch
>
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3405) SSH action shows empty error Message and Error code
[
https://issues.apache.org/jira/browse/OOZIE-3405?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16924366#comment-16924366
]
Julia Kinga Marton commented on OOZIE-3405:
---
[~matijhs], I can see in the review that you have uploaded multiple patches.
Can you please upload your patch here as well, to trigger the precommit script?
> SSH action shows empty error Message and Error code
> ---
>
> Key: OOZIE-3405
> URL: https://issues.apache.org/jira/browse/OOZIE-3405
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: 5.1.0
>Reporter: Peter Orova
>Assignee: Mate Juhasz
>Priority: Minor
> Fix For: trunk
>
> Attachments: OOZIE-3405-V1.patch
>
>
> Currently, when an SSH action fails the only message that is returned is the
> Status. Neither the {{error Message}} nor {{Error code}} fields are filled.
> This makes reporting on the causes of SSH Action failures via Oozie highly
> impractical: the only meaningful bit of information there is on a failed SSH
> Action is the Status.
> The Status is filled based on the presence (or lack of) the {{.error file}}
> that is produced in case the user submitted script returns with any other
> value than 0.
> {noformat}
> SshActionExecutor#getActionStatus
> ...
> String outFile = getRemoteFileName(context, action, "error", false, true);
> String checkErrorCmd = SSH_COMMAND_BASE + action.getTrackerUri() + " ls " +
> outFile;
> int retVal = getReturnValue(checkErrorCmd);
> ...
> {noformat}
>
> User requirement is to provide some more detailed information on the
> success/failure of the user-submitted script. That could be at a minimum the
> return value, optionally the last ~1K of the stderr that is drained. This
> information could then be communicated via {{errorMessage}} and {{ErrorCode}}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3540) Use StringBuilder instead of StringBuffer if concurrent access is not required
[
https://issues.apache.org/jira/browse/OOZIE-3540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16920666#comment-16920666
]
Julia Kinga Marton commented on OOZIE-3540:
---
[~zsombor], unfortunately the spotbugs check is buggy and there are a lot of
old bugs reported as new ones. (we have an open Jira for replacing the spotbugs
diff jar OOZIE-3445).
The "newly" reported bug in this case is an old one in the {{ShellMain.java}}
file [line 92].
> Use StringBuilder instead of StringBuffer if concurrent access is not required
> --
>
> Key: OOZIE-3540
> URL: https://issues.apache.org/jira/browse/OOZIE-3540
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: trunk
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Minor
> Fix For: 5.2.0
>
> Attachments: OOZIE-3540.patch
>
>
> StringBuffer is an old relic from the distant past, Oozie shouldnt' need to
> use it, unless concurrent access is expected.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (OOZIE-3360) Upgrade to Log4j2
[ https://issues.apache.org/jira/browse/OOZIE-3360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3360: - Assignee: (was: Julia Kinga Marton) > Upgrade to Log4j2 > - > > Key: OOZIE-3360 > URL: https://issues.apache.org/jira/browse/OOZIE-3360 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Priority: Major > > The Log4j™ 1.x logging framework has reached its end of life (EOL) and is no > longer officially supported. > We should upgrade the Oozie server from Log4j1 to Log4j2 and add Log4j2 > support for Sharelib. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (OOZIE-3359) Check for Process#waitFor() usage and fix it in order to avoid indefinite waiting
[
https://issues.apache.org/jira/browse/OOZIE-3359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3359:
-
Assignee: (was: Julia Kinga Marton)
> Check for Process#waitFor() usage and fix it in order to avoid indefinite
> waiting
> -
>
> Key: OOZIE-3359
> URL: https://issues.apache.org/jira/browse/OOZIE-3359
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Priority: Major
> Fix For: 5.2.0
>
>
> {{Process#waitFor()}} will block until the process finishes. There are
> situations where this will wait indefinitely. A similar case was in
> SshActionExecutor, fixed in OOZIE-3354.
> Let's check the code for other usages, and fix it.
> Maybe we should check if we can use somehow the Process#waitFor(long timeout,
> TimeUnit timeUnit) instead of it. The tricky part is that this method is
> available only in Java 8.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3536) oozie-main(pom.xml) plugin maven-javadoc-plugin upgrade version caused configuration can't find the Tag
[
https://issues.apache.org/jira/browse/OOZIE-3536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918479#comment-16918479
]
Julia Kinga Marton commented on OOZIE-3536:
---
[~nobigo], According to the [stackoverflow
link|https://stackoverflow.com/questions/52547306/maven-javadoc-plugin-not-accepting-additionalparam-xdoclintnone-additionalpa]
you have shared, the javadoc generation should fail because of this issue, but
{{mvn javadoc:javadoc }}succeeds.
Can you please share what failed, and how can we reproduce the issue?
> oozie-main(pom.xml) plugin maven-javadoc-plugin upgrade version caused
> configuration can't find the Tag
> --
>
> Key: OOZIE-3536
> URL: https://issues.apache.org/jira/browse/OOZIE-3536
> Project: Oozie
> Issue Type: Bug
> Components: build
>Affects Versions: trunk, 5.1.0, 5.2.0
>Reporter: duan xiong
>Assignee: duan xiong
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3536-001.patch
>
>
> oozie-main(pom.xml) plugin maven-javadoc-plugin upgrade 3.0.1version caused
> configuration can't find the Tag . In early version
> 2.10.4,this plugin has this tag
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3468) [build] Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918458#comment-16918458
]
Julia Kinga Marton commented on OOZIE-3468:
---
hank you [~asalamon74] for the contribution! +1, committed to master
> [build] Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch,
> OOZIE-3468-07.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (OOZIE-3468) [build] Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16918458#comment-16918458
]
Julia Kinga Marton edited comment on OOZIE-3468 at 8/29/19 9:53 AM:
Thank you [~asalamon74] for the contribution! +1, committed to master
was (Author: kmarton):
hank you [~asalamon74] for the contribution! +1, committed to master
> [build] Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch,
> OOZIE-3468-07.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (OOZIE-3468) [build] Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3468:
--
Summary: [build] Use modernizer plugin (was: Use modernizer plugin)
> [build] Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch,
> OOZIE-3468-07.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902065#comment-16902065
]
Julia Kinga Marton commented on OOZIE-3468:
---
[~asalamon74], I have some small comments related the actual patch:
* in the modernizer script the PATCHFILE option is not used. can you please
remove it?
* _"REPORT+=("\{color:red}-1\{color} the patch introduces ${newErrorInClass}
new warnings in ${class_name}")",_ please replace warning with warning(s).
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891862#comment-16891862
]
Julia Kinga Marton edited comment on OOZIE-3468 at 8/7/19 11:59 AM:
[~asalamon74] can you please check your code with ShellCheck and fix the
reported issues?
was (Author: kmarton):
[~asalamon74] can you please check tour code with ShellCheck and fix the
reported issues?
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (OOZIE-3535) mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full effect
[
https://issues.apache.org/jira/browse/OOZIE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3535:
--
Attachment: OOZIE-3535-001.patch
> mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full
> effect
>
>
> Key: OOZIE-3535
> URL: https://issues.apache.org/jira/browse/OOZIE-3535
> Project: Oozie
> Issue Type: Bug
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3535-001.patch
>
>
> mapreduce.job.acl-view-job property in Oozie workflow.xml will be ignored if
> only {{yarn.acl.enable}} = true, but {{mapreduce.cluster.acls.enabled}} =
> false.
> This is caused by the following check:
> [https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/YarnACLHandler.java#L39]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (OOZIE-3535) mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full effect
Julia Kinga Marton created OOZIE-3535:
-
Summary: mapreduce.job.acl-view-job property in Oozie workflow.xml
not taking full effect
Key: OOZIE-3535
URL: https://issues.apache.org/jira/browse/OOZIE-3535
Project: Oozie
Issue Type: Bug
Reporter: Julia Kinga Marton
Assignee: Julia Kinga Marton
mapreduce.job.acl-view-job property in Oozie workflow.xml will be ignored if
only {{yarn.acl.enable}} = true, but {{mapreduce.cluster.acls.enabled}} = false.
This is caused by the following check:
[https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/YarnACLHandler.java#L39]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3534) Refactor ActionExecutorTestCase to not extend HCatTestCase
[ https://issues.apache.org/jira/browse/OOZIE-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16900039#comment-16900039 ] Julia Kinga Marton commented on OOZIE-3534: --- The task is not so trivial, because after OOZIE-2287 a lot of classes got dependent on this HCatTestCase even if only a few ones really needs it. > Refactor ActionExecutorTestCase to not extend HCatTestCase > -- > > Key: OOZIE-3534 > URL: https://issues.apache.org/jira/browse/OOZIE-3534 > Project: Oozie > Issue Type: Improvement > Components: tests >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > ActionExecutorTestCase extends HCatTestCase but there are only two test cases > where we need HCatalog related settings as well. > Let's try to refactor this part and make the ActionExecturTestCase > independent from the HCatalog part. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3214) Allow configurable timezone for coordinators
[
https://issues.apache.org/jira/browse/OOZIE-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3214:
-
Assignee: (was: Julia Kinga Marton)
> Allow configurable timezone for coordinators
>
>
> Key: OOZIE-3214
> URL: https://issues.apache.org/jira/browse/OOZIE-3214
> Project: Oozie
> Issue Type: New Feature
> Components: coordinator
>Affects Versions: trunk
>Reporter: Julia Kinga Marton
>Priority: Major
>
> Right now in case of coordinators only daylight saving time corrections are
> applied, the processing timezone for the coordinators is always the Oozie
> processing timezone.
> It would be more transparent to have an option for changing the {{timezone}}
> attribute itself, such as an additional attribute in the {{coordinator.xml}}
> (meaning a new version of {{coordinator.xsd}}). I would make this option
> switched off by default(to have the actual behavior).
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3177) Unclear ooziedb -sqlFile option behaviour
[ https://issues.apache.org/jira/browse/OOZIE-3177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3177: - Assignee: (was: Julia Kinga Marton) > Unclear ooziedb -sqlFile option behaviour > -- > > Key: OOZIE-3177 > URL: https://issues.apache.org/jira/browse/OOZIE-3177 > Project: Oozie > Issue Type: Bug > Components: scripts >Reporter: Julia Kinga Marton >Priority: Minor > > There are two scenarios when running ooziedb.sh with -sqlscript option can > lead to misunderstandings: > full command: ooziedb.sh upgrade -sqlfile oozie-upgrade.sql > # if there are no changes, the following information is printed out: "The > SQL commands have been written to: oozie-upgrade.sql", however if there is no > SQL command, the file will be not created. > # If there are changes and the file is created, it is stored in the /tmp/ > directory. In the log message it would be good to print out the full path to > the file, not only the file name. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-2231) Upgrade curator to latest version 2.13.0
[ https://issues.apache.org/jira/browse/OOZIE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-2231: - Assignee: (was: Julia Kinga Marton) > Upgrade curator to latest version 2.13.0 > > > Key: OOZIE-2231 > URL: https://issues.apache.org/jira/browse/OOZIE-2231 > Project: Oozie > Issue Type: Bug > Components: HA >Reporter: Purshotam Shah >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-2231-00.patch, OOZIE-2231-01.patch, > OOZIE-2231-02.patch, OOZIE-2231-02.patch, OOZIE-2231-03.patch, > OOZIE-2231-04.patch, OOZIE-2231-05.patch, OOZIE-2231-06.patch > > > It have some fix related to InterProcessReadWriteLock, ChildReaper, > LeaderSelector which we use. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3350) Forkjoin validation does not fail if a node is reachable from two different forks
[
https://issues.apache.org/jira/browse/OOZIE-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3350:
-
Assignee: (was: Julia Kinga Marton)
> Forkjoin validation does not fail if a node is reachable from two different
> forks
> -
>
> Key: OOZIE-3350
> URL: https://issues.apache.org/jira/browse/OOZIE-3350
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 4.3.1
>Reporter: wang jinyin
>Priority: Major
> Fix For: trunk
>
> Original Estimate: 120h
> Remaining Estimate: 120h
>
> when "multiple ok to same node" under decision node, forkjoin validation
> error.
>
> such as below example, 'action_C' and 'action_D' both transition to
> 'action_E'.
> But, because they are under same topDecisionParent 'decision_A', validator
> will not throw any exception.
>
> {quote}
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3136) Upgrade from Log4j 1.x to 2.x
[
https://issues.apache.org/jira/browse/OOZIE-3136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3136:
-
Assignee: (was: Julia Kinga Marton)
> Upgrade from Log4j 1.x to 2.x
> -
>
> Key: OOZIE-3136
> URL: https://issues.apache.org/jira/browse/OOZIE-3136
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Attila Sasvari
>Priority: Major
>
> {{5 August 2015 --The Apache Logging Services™ Project Management Committee
> (PMC) has announced that the Log4j™ 1.x logging framework has reached its end
> of life (EOL) and is no longer officially supported.}}
> https://blogs.apache.org/foundation/entry/apache_logging_services_project_announces
> We should upgrade from Log4j 1.x to 2.x . Perhaps we could use slf4j .
> Related tickets: MAPREDUCE-6983, HADOOP-12956, OOZIE-3135
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3336) [persistence] Refactor entity classes to feature PK, FK, and UQ constraints
[
https://issues.apache.org/jira/browse/OOZIE-3336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3336:
-
Assignee: (was: Julia Kinga Marton)
> [persistence] Refactor entity classes to feature PK, FK, and UQ constraints
> ---
>
> Key: OOZIE-3336
> URL: https://issues.apache.org/jira/browse/OOZIE-3336
> Project: Oozie
> Issue Type: Improvement
> Components: core
>Affects Versions: 5.0.0
>Reporter: Andras Piros
>Priority: Major
> Fix For: 5.2.0
>
>
> When an Oozie database grows substantial in size, let's say, over a few
> hundred thousands of {{WorkflowActionBean}}, {{CoordinatorActionBean}}
> instances, we face a couple of performance issues. Here is an analysis why.
> Current Oozie JPA {{@Entity}} usage, and the resulting database DDL, suffers
> from a couple of drawback from a performance point of view:
> * {{@Id}} fields are {{String}}:
> ** leaving no space for database primary key indices to work effectively
> ** those values are calculated in case of {{WorkflowActionBean}},
> {{CoordinatorActionBean}}, and {{BundleActionBean}} instances
> * no foreign constraint is set from {{WorkflowActionBean}} to
> {{WorkflowJobBean}}, from {{CoordinatorActionBean}} to
> {{CoordinatorJobBean}}, or from {{BundleActionBean}} to {{BundleJobBean}}
> instances:
> ** have to assess JPA queries discovering parent-child relationships by hand
> ** no database indices are created, and hence, those queries that contain any
> {{JOIN}} instances are slower
> * no use of unique constraints whatsoever
> * JPA queries are created by hand instead of relying on OpenJPA
> * JPA entities are filled by hand instead of relying on OpenJPA
> Following enhancements are necessary:
> # keeping the existing {{String compositeId}} fields, let's break down the
> contents to following new fields:
> ## {{@Id long id}} - an auto-increment value that is unique across Oozie
> database
> ## {{long currentSequence}} - the sequence number of the current run since
> last Oozie server restart. The first part of the {{compositeId}}
> ## {{Timestamp serverStartupTimestamp}} - the timestamp when the Oozie server
> was last started. The second part of the {{compositeId}}
> ## {{String serverName}} - the third part of the {{compositeId}}
> ## {{String name}} - the fourth and last part of the {{compositeId}}
> ## {{compositeId}} might be calculated when an entity is loaded / persisted,
> and then stored
> # FK constraints:
> ## {{@OneToMany}} fields where we have a list of child references inside
> parent
> ## {{@ManyToOne}} fields where we have a parent reference inside child
> ## pay attention to {{FetchType}}, most of the times {{LAZY}} will be needed
> ## the containment fields should not be {{@Transient}} anymore
> # UQ constraints:
> ## on {{currentSequence}} and {{serverStartupTimestamp}}
> ## on {{currentSequence}} and {{name}}
> # new JPQL queries:
> ## to cover changed parent-child relationships
> ## to get use of each disassembled part of {{originalId}} when doing e.g.
> filtering
> # let JPA fill entities instead performing this by hand
> Following enhancements can be considered as nice-to-have:
> * upgrade to an OpenJPA version that features JPA 2.1's composite indexing
> capability
> * see whether to have an optimistic locking field using {{@Version}} instead
> of ZooKeeper based pessimistic locking would increase High Availability
> characteristics
> * refactor also SLA related entity classes
> It's necessary to have performance benchmarks with some database types like
> MySQL/MariaDB, and PostgreSQL before and after the changes for following use
> cases:
> * {{CoordinatorJobBean}} and {{WorkflowJobBean}} instances up to millions
> * {{CoordinatorActionBean}} and {{WorkflowActionBean}} instances up to tens
> of millions
> * performance for JPQLs that get a list of entities
> * performance of persisting a new entity
> * performance of querying lists of entities based on popular / possible
> filters like the ones used by {{VxJobsServlet}}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3469) Extract common methods, fields from oozie-core to a new oozie-common module
[ https://issues.apache.org/jira/browse/OOZIE-3469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3469: - Assignee: (was: Julia Kinga Marton) > Extract common methods, fields from oozie-core to a new oozie-common module > --- > > Key: OOZIE-3469 > URL: https://issues.apache.org/jira/browse/OOZIE-3469 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Priority: Major > > Oozie sharelib needs oozie-core as dependency, what will bring in a lot of > transitive dependencies, what may cause conflicts and we need to exclude them > one by one. In a lot of cases we need only some constants from oozie-core. > Let's investigate whether we can extract this common fields, > (methods/classes) into a new common module. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3135) Configure log4j2 in SqoopMain
[
https://issues.apache.org/jira/browse/OOZIE-3135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-3135:
-
Assignee: (was: Julia Kinga Marton)
> Configure log4j2 in SqoopMain
> -
>
> Key: OOZIE-3135
> URL: https://issues.apache.org/jira/browse/OOZIE-3135
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: 5.0.0b1
>Reporter: Attila Sasvari
>Priority: Major
> Attachments: OOZIE-3135-001.patch
>
>
> In Hadoop 3, MAPREDUCE-6983 switched to use slfj4 & log4j2 in
> {{org.apache.hadoop.mapreduce.Job}} (that prints out MR job id-s needed for
> Oozie). We need to setup log4j accordingly (it is also related to
> HADOOP-12956).
> Without proper configuration in the Sqoop action, we won't be able to get
> external job id-s (SqoopActionExecutor unit tests and real action would be
> also affected).
>
> [The API for Log4j 2 is not compatible with Log4j
> 1.x|https://logging.apache.org/log4j/2.x/], but we will need to support both
> hadoop 2 and hadoop 3 profiles for a while.
> We could use reflection to determine the type of the logger object in
> {{org.apache.hadoop.mapreduce.Job}} and configure log4j settings based on it,
> but there might be a better way.
> For example we could do something like this:
> - add a new method for configuring log4j2:
> {code}
> private String setUpSqoopLog4J2(final String rootLogLevel) throws
> IOException {
> System.out.println("Setting up log4j2");
> final String logFile = getSqoopLogFile();
> final File log4j2Xml = new File(SQOOP_LOG4J2_XML);
> try (Writer writer = new FileWriter(log4j2Xml))
> {
> final String logj2SettingsXml = " encoding=\"UTF-8\"?>\n" +
> "\n" +
> "\n" +
> " target=\"SYSTEM_OUT\">\n" +
> " [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> "\n" +
> " "\"> \n" +
> " [%t] %-5level %logger{36} - %msg%n\"/>\n" +
> " \n" +
> "\n" +
> "\n" +
> " "\">\n" +
> "\n" +
> "\n" +
> "\n" +
> "\n" +
> "";
> writer.write(logj2SettingsXml);
> }
> System.out.printf("log4j2 configuration file created at %s%n",
> log4j2Xml.getAbsolutePath());
> final LoggerContext context = (LoggerContext)
> LogManager.getContext(false);
> context.setConfigLocation(log4j2Xml.toURI()); // forces log4j2
> reconfiguration
> return logFile;
> }
> {code}
> and call it in the {{run()}} method if the mapreduce client is using slf4j
> for logging:
> {code}
> String logFile;
> // MAPREDUCE-6983 switches to slfj4 & log4j2. Need to setup log4j
> accordingly
> if
> (org.apache.hadoop.mapreduce.Job.class.getDeclaredField("LOG").getType().
> isAssignableFrom(org.slf4j.Logger.class)) {
> logFile = setUpSqoopLog4J2(rootLogLevel);
> }
> else {
> logFile = setUpSqoopLog4J(rootLogLevel, logLevel);
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Assigned] (OOZIE-3356) Add blacklist for EL evaluation
[ https://issues.apache.org/jira/browse/OOZIE-3356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3356: - Assignee: (was: Julia Kinga Marton) > Add blacklist for EL evaluation > - > > Key: OOZIE-3356 > URL: https://issues.apache.org/jira/browse/OOZIE-3356 > Project: Oozie > Issue Type: Improvement >Affects Versions: trunk >Reporter: Julia Kinga Marton >Priority: Major > > OOZIE-1580 fixed that the EL variables included using got fixed, > but there are cases when we pass some *-site.xml, and the variables have > different format. > For example when talking to hbase, we can pass the hbase-site.xml using > tag, and if there are some variables defined, we need to modify > them manually. > By adding a blacklist, when we don't want to evaluate the EL expressions, we > can avoid this manual interaction. > By default only the *-site.xml files should be ignored, but this should be > configurable. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3416) Incorporate dbd and dbd testing into Apache Oozie/tools
[ https://issues.apache.org/jira/browse/OOZIE-3416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3416: - Assignee: (was: Julia Kinga Marton) > Incorporate dbd and dbd testing into Apache Oozie/tools > --- > > Key: OOZIE-3416 > URL: https://issues.apache.org/jira/browse/OOZIE-3416 > Project: Oozie > Issue Type: Task > Components: tools >Affects Versions: 5.1.0 >Reporter: Julia Kinga Marton >Priority: Major > > [~daniel.becker] implemented a dockerised integration testing framework. > It consists of two repositories: > * dbd - [https://github.com/d-becker/dbd]This tool’s aim is to make it easy > to set up a working dockerized big data infrastructure where the versions of > the components (Hadoop, Hive, Oozie etc.) can be set individually, and even > unreleased snapshot builds can be used. > * oozie-dbd-testing - [https://github.com/d-becker/oozie-dbd-testing] This > is the repository that contains the scripts and the Jenkins job definition > used to test Oozie. It runs the Oozie examples and the fluent job examples > and produces a report. To set up a Jenkins job, use the "Multibranch > pipeline" project type. > Now, that this tools are working, we should integrate them in the Apache > Oozie repository next to the other tools. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Assigned] (OOZIE-3301) Update NOTICE file
[ https://issues.apache.org/jira/browse/OOZIE-3301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton reassigned OOZIE-3301: - Assignee: (was: Julia Kinga Marton) > Update NOTICE file > -- > > Key: OOZIE-3301 > URL: https://issues.apache.org/jira/browse/OOZIE-3301 > Project: Oozie > Issue Type: Bug >Affects Versions: 5.0.0 >Reporter: Attila Sasvari >Priority: Major > Attachments: OOZIE-3301-001.patch, OOZIE-3301-002-reupload.patch, > THIRD-PARTY.txt > > > Oozie NOTICE file is not up to date. Please overview and update it as per > http://www.apache.org/licenses/LICENSE-2.0.html#redistribution > For example: Oozie uses embedded jetty, however Eclipse license > (http://www.eclipse.org/jetty/documentation/9.3.x/embedding-jetty.html) is > not listed in > [NOTICE.txt|https://github.com/apache/oozie/blob/a299d4a6d435a2c92cd1d0ffce7f35a2ef8d639b/NOTICE.txt#L9]. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (OOZIE-3534) Refactor ActionExecutorTestCase to not extend HCatTestCase
Julia Kinga Marton created OOZIE-3534: - Summary: Refactor ActionExecutorTestCase to not extend HCatTestCase Key: OOZIE-3534 URL: https://issues.apache.org/jira/browse/OOZIE-3534 Project: Oozie Issue Type: Improvement Components: tests Reporter: Julia Kinga Marton Assignee: Julia Kinga Marton ActionExecutorTestCase extends HCatTestCase but there are only two test cases where we need HCatalog related settings as well. Let's try to refactor this part and make the ActionExecturTestCase independent from the HCatalog part. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3496) Upgrade ActiveMQ to 5.15.9
[ https://issues.apache.org/jira/browse/OOZIE-3496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16898648#comment-16898648 ] Julia Kinga Marton commented on OOZIE-3496: --- Thank you [~asalamon74]! +1, committed to master > Upgrade ActiveMQ to 5.15.9 > -- > > Key: OOZIE-3496 > URL: https://issues.apache.org/jira/browse/OOZIE-3496 > Project: Oozie > Issue Type: Bug >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-3496-01-wip.patch, OOZIE-3496-02-wip.patch, > OOZIE-3496-03-wip.patch, OOZIE-3496-04.patch > > > Although we only use it for tests we should still upgrade ActiveMQ to the > latest 5.15.9 version from the current 5.15.3 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891862#comment-16891862
]
Julia Kinga Marton commented on OOZIE-3468:
---
[~asalamon74] can you please check tour code with ShellCheck and fix the
reported issues?
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891716#comment-16891716 ] Julia Kinga Marton commented on OOZIE-3504: --- The test error is unrelated, however I retriggered the build. Let's wait for the new results. > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3504-001.patch > > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3504: -- Attachment: OOZIE-3504-001.patch > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3504-001.patch > > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890971#comment-16890971 ] Julia Kinga Marton commented on OOZIE-3504: --- TestHCatCredentials is failing because of classloading issues: it seems that PowerMock has a custom classloader what nows nothing about the modules in JDK11. We can instruct Powermock to let the parent classloader to load some classes by using @PowerMockIgnore annotation. > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3504-001.patch > > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890889#comment-16890889 ] Julia Kinga Marton commented on OOZIE-3504: --- OOZIE-3528 solved the TestScriptLanguageActionExecutor failure, but TestHCatCredentials is still failing. > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890119#comment-16890119 ] Julia Kinga Marton commented on OOZIE-3528: --- testActionKillCommandDate:org.apache.oozie.command.coord.TestCoordActionsKillXCommand passed locally, so I am retriggering the precommit build > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890016#comment-16890016 ] Julia Kinga Marton edited comment on OOZIE-3528 at 7/22/19 9:16 AM: [~asalamon74] below you can find my answers to your questions: * I will check with JDK11 during OOZIE-3504. In this fix my purpose was to do the upgrade, and to make everything with the new versions using JDK8 * You are right, we should use the newer version from branch 2. I have run some tests manually with 2.28.2, they passed, let's see what the precommit will show. * variable creation: done was (Author: kmarton): [~asalamon74] below you can find my answers to your questions: * I will check with JDK11 during OOZIE-3504. In this fix my purpose was to do the upgrade, and to make everything with the new versions using JDK8 * You are right, we should use the newer version from line2. I have run some tests manually with 2.28.2, they passed, let's see what the precommit will show. * variable creation: done > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890016#comment-16890016 ] Julia Kinga Marton commented on OOZIE-3528: --- [~asalamon74] below you can find my answers to your questions: * I will check with JDK11 during OOZIE-3504. In this fix my purpose was to do the upgrade, and to make everything with the new versions using JDK8 * You are right, we should use the newer version from line2. I have run some tests manually with 2.28.2, they passed, let's see what the precommit will show. * variable creation: done > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-003.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > OOZIE-3528-003.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (OOZIE-2882) Rerun workflow fails Error: E0404
[
https://issues.apache.org/jira/browse/OOZIE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton resolved OOZIE-2882.
---
Resolution: Fixed
Fix Version/s: 5.2.0
This issue was fixed with OOZIE-3265
> Rerun workflow fails Error: E0404
> -
>
> Key: OOZIE-2882
> URL: https://issues.apache.org/jira/browse/OOZIE-2882
> Project: Oozie
> Issue Type: Improvement
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Major
> Fix For: 5.2.0
>
>
> Only one of the properties are allowed [oozie.wf.rerun.skip.nodes OR
> oozie.wf.rerun.failnodes]
> Reproduction:
> 1. Create a workflow with more than 1 node. Eg: Fork - with three parallel
> shell actions. Make sure one of them fails
> 2. Rerun with 'oozie.wf.rerun.failnodes' set.
> 3. Rerun again with 'oozie.wf.rerun.skip.nodes' and check 'Skip all
> successful nodes'.
> You will get the following error.
> Error: E0404 : E0404: Only one of the properties are allowed
> [oozie.wf.rerun.skip.nodes OR oozie.wf.rerun.failnodes]
> When a user reruns a workflow job with oozie.wf.rerun.failnode=true and if
> the job fails in subsequent steps, we do not have an option to resubmit the
> workflow using oozie.wf.rerun.skip.node=action1,action2 to allow submission
> from predecessor steps.
> Currently, once the workflow fails and one of the rerun options is used for
> job rerun it gets merged and there is no way to override like regular oozie
> configurations or variables.
> We have a few options:
> 1. If fail.nodes and skip.nodes are specified at the same time (or one of
> them was carried over from a previous wf run), we can add {generate
> skip.nodes by discovering nodes that did not fail} union {skip.nodes}
> 2. Add a way to remove properties (this is also is potentially helpful for
> other use cases)
> 3. The "newest" property (oozie.wf.rerun.skip.nodes or
> oozie.wf.rerun.failnodes) takes priority and the previous is ignored
> 4. Make oozie.wf.rerun.skip.nodes or oozie.wf.rerun.failnodes somehow not
> persist in the DB
> Part of this JIRA would be to figure out which is the best option.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-002.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, OOZIE-3528-002.patch, > WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-001.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: OOZIE-3528-001.patch, WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3265: -- Attachment: OOZIE-3265-007.patch > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: Julia Kinga Marton >Priority: Minor > Attachments: OOZIE-3265-004.patch, OOZIE-3265-005.patch, > OOZIE-3265-006.patch, OOZIE-3265-007.patch, OOZIE-3265-v1.patch, > OOZIE-3265-v2.patch, OOZIE-3265-v3.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-2755) Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is zero
[
https://issues.apache.org/jira/browse/OOZIE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886894#comment-16886894
]
Julia Kinga Marton commented on OOZIE-2755:
---
committed to master
> Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is
> zero
> --
>
> Key: OOZIE-2755
> URL: https://issues.apache.org/jira/browse/OOZIE-2755
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Dongying Jiao
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-2755-01.patch, OOZIE-2755-02.patch
>
>
> Setting up Oozie HA using virtual IP, {{server-1}} and {{server-2}}
> (active-active), when we take down {{server-1}} any Oozie job submitted fails
> with below stacktrace. If both are up , there is no issue.
> {code:java}
> ERROR RecoveryService$RecoveryRunnable:517 - SERVER[XXX] USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Exception, / by zero
> java.lang.ArithmeticException: / by zero
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.checkJobIdForServer(ZKJobsConcurrencyService.java:167)
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.isJobIdForThisServer(ZKJobsConcurrencyService.java:129)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.runWFRecovery(RecoveryService.java:362)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.run(RecoveryService.java:146)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-2755) Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is zero
[
https://issues.apache.org/jira/browse/OOZIE-2755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16886890#comment-16886890
]
Julia Kinga Marton commented on OOZIE-2755:
---
I agree with [~asalamon74] that simply returning true may cause other issues.
+1 for this specific warning approach.
> Oozie HA: ZKJobsConcurrencyService throws runtime exception when numOozies is
> zero
> --
>
> Key: OOZIE-2755
> URL: https://issues.apache.org/jira/browse/OOZIE-2755
> Project: Oozie
> Issue Type: Bug
> Components: HA
>Reporter: Dongying Jiao
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-2755-01.patch, OOZIE-2755-02.patch
>
>
> Setting up Oozie HA using virtual IP, {{server-1}} and {{server-2}}
> (active-active), when we take down {{server-1}} any Oozie job submitted fails
> with below stacktrace. If both are up , there is no issue.
> {code:java}
> ERROR RecoveryService$RecoveryRunnable:517 - SERVER[XXX] USER[-] GROUP[-]
> TOKEN[-] APP[-] JOB[-] ACTION[-] Exception, / by zero
> java.lang.ArithmeticException: / by zero
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.checkJobIdForServer(ZKJobsConcurrencyService.java:167)
> at
> org.apache.oozie.service.ZKJobsConcurrencyService.isJobIdForThisServer(ZKJobsConcurrencyService.java:129)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.runWFRecovery(RecoveryService.java:362)
> at
> org.apache.oozie.service.RecoveryService$RecoveryRunnable.run(RecoveryService.java:146)
> at
> org.apache.oozie.service.SchedulerService$2.run(SchedulerService.java:175)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16885147#comment-16885147 ] Julia Kinga Marton commented on OOZIE-3265: --- Sure [~asalamon74], done > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: Julia Kinga Marton >Priority: Minor > Attachments: OOZIE-3265-004.patch, OOZIE-3265-005.patch, > OOZIE-3265-006.patch, OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, > OOZIE-3265-v3.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Comment Edited] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883758#comment-16883758 ] Julia Kinga Marton edited comment on OOZIE-3528 at 7/12/19 12:26 PM: - *DO NOT MERGE WIP_OOZIE-3528.patch*! I expect to have a couple of test failure. I have uploaded it to check the precommit result. was (Author: kmarton): DO NOT MERGE YET! I expect to have a couple of test failure. I have uploaded it to check the precommit result. > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: WIP_OOZIE-3528.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883758#comment-16883758 ] Julia Kinga Marton commented on OOZIE-3528: --- DO NOT MERGE YET! I expect to have a couple of test failure. I have uploaded it to check the precommit result. > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > Attachments: WIP_OOZIE-3528.patch > > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: (was: OOZIE-3528-001.patch) > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Attachment: OOZIE-3528-001.patch > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to Powermock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Description: PowerMock 1 does not support JDK11. If we want to . support it we will need to migrate to PoweMock 2 > Migrate to Powermock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3528) Migrate to PowerMock 2 and Mockito2
[ https://issues.apache.org/jira/browse/OOZIE-3528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3528: -- Summary: Migrate to PowerMock 2 and Mockito2 (was: Migrate to Powermock 2 and Mockito2) > Migrate to PowerMock 2 and Mockito2 > --- > > Key: OOZIE-3528 > URL: https://issues.apache.org/jira/browse/OOZIE-3528 > Project: Oozie > Issue Type: Improvement >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > PowerMock 1 does not support JDK11. If we want to . support it we will need > to migrate to PoweMock 2 -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16883748#comment-16883748 ] Julia Kinga Marton commented on OOZIE-3504: --- It seems that Powermock2 supports JDK11, so I have opened a Jira to make this upgrade: OOZIE-3528 > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (OOZIE-3528) Migrate to Powermock 2 and Mockito2
Julia Kinga Marton created OOZIE-3528: - Summary: Migrate to Powermock 2 and Mockito2 Key: OOZIE-3528 URL: https://issues.apache.org/jira/browse/OOZIE-3528 Project: Oozie Issue Type: Improvement Reporter: Julia Kinga Marton Assignee: Julia Kinga Marton -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-3265) properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together
[ https://issues.apache.org/jira/browse/OOZIE-3265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3265: -- Attachment: OOZIE-3265-006.patch > properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear > together > -- > > Key: OOZIE-3265 > URL: https://issues.apache.org/jira/browse/OOZIE-3265 > Project: Oozie > Issue Type: Task >Affects Versions: 5.0.0 >Reporter: TIAN XING >Assignee: Julia Kinga Marton >Priority: Minor > Attachments: OOZIE-3265-004.patch, OOZIE-3265-005.patch, > OOZIE-3265-006.patch, OOZIE-3265-v1.patch, OOZIE-3265-v2.patch, > OOZIE-3265-v3.patch, rerun.patch > > > Currently when you re-run a workflow with property "oozie.wf.rerun.failnodes" > set to true, > you can no longer re-run it again with "oozie.wf.rerun.skip.nodes" property > specified, even if you set "oozie.wf.rerun.failnodes" to false. > This kind of limitation is not reasonable. There is only one case where > "oozie.wf.rerun.failnodes" is true and "oozie.wf.rerun.skip.nodes" is not > null or empty, that should be disallowed. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (OOZIE-2836) Remove .ps1 and .cmd scripts
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-2836:
--
Attachment: OOZIE-2836-002.patch
> Remove .ps1 and .cmd scripts
>
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-2836-002.patch, OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-2836) Remove .ps1 and .cmd scripts
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-2836:
--
Summary: Remove .ps1 and .cmd scripts (was: Fix ps1 and cmd scripts to
make Oozie runnable on Windows)
> Remove .ps1 and .cmd scripts
>
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (OOZIE-2836) Fix ps1 and cmd scripts to make Oozie runnable on Windows
[
https://issues.apache.org/jira/browse/OOZIE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton reassigned OOZIE-2836:
-
Assignee: Julia Kinga Marton
> Fix ps1 and cmd scripts to make Oozie runnable on Windows
> -
>
> Key: OOZIE-2836
> URL: https://issues.apache.org/jira/browse/OOZIE-2836
> Project: Oozie
> Issue Type: Bug
>Reporter: Attila Sasvari
>Assignee: Julia Kinga Marton
>Priority: Minor
> Attachments: OOZIE-2836-01.patch
>
>
> With OOZIE-2666 Oozie was migrated from Tomcat to embedded jetty. Distro
> shell scripts were updated, but Windows scripts were not adjusted in
> {{distro/src/main/bin}}.
> It would be nice if someone with a Windows machine could fix the following
> scripts:
> - oozie-setup.cmd
> - oozie-setup.ps1
> - oozie-sys.cmd
> - oozie.cmd
> - oozied.cmd
> and verify that Oozie works on Windows.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16878327#comment-16878327
]
Julia Kinga Marton commented on OOZIE-3468:
---
+1 for adding the modrnizer to our precommit script.
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3506) Flaky test TestOozieRollingPolicy
[
https://issues.apache.org/jira/browse/OOZIE-3506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877810#comment-16877810
]
Julia Kinga Marton commented on OOZIE-3506:
---
Thank you [~asalamon74]! +1, committed to master
> Flaky test TestOozieRollingPolicy
> -
>
> Key: OOZIE-3506
> URL: https://issues.apache.org/jira/browse/OOZIE-3506
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Minor
> Attachments: OOZIE-3506-01.patch
>
>
> {{TestOozieRollingPolicy}} sometimes fails with the following error message:
> {noformat}2019-05-30 13:15:21 [INFO] Running
> org.apache.oozie.util.TestOozieRollingPolicy
> 2019-05-30 13:15:21 [ERROR] Tests run: 3, Failures: 1, Errors: 0, Skipped: 0,
> Time elapsed: 87.025 s <<< FAILURE! - in
> org.apache.oozie.util.TestOozieRollingPolicy
> 2019-05-30 13:15:21 [ERROR]
> testDeletingOldFiles(org.apache.oozie.util.TestOozieRollingPolicy) Time
> elapsed: 60.337 s <<< FAILURE!
> 2019-05-30 13:15:21 junit.framework.AssertionFailedError
> 2019-05-30 13:15:21 at junit.framework.Assert.fail(Assert.java:55)
> 2019-05-30 13:15:21 at junit.framework.Assert.assertTrue(Assert.java:22)
> 2019-05-30 13:15:21 at junit.framework.Assert.assertTrue(Assert.java:31)
> 2019-05-30 13:15:21 at
> junit.framework.TestCase.assertTrue(TestCase.java:201)
> 2019-05-30 13:15:21 at
> org.apache.oozie.util.TestOozieRollingPolicy._testDeletingOldFiles(TestOozieRollingPolicy.java:125)
> 2019-05-30 13:15:21 at
> org.apache.oozie.util.TestOozieRollingPolicy._testDeletingOldFiles(TestOozieRollingPolicy.java:59)
> 2019-05-30 13:15:21 at
> org.apache.oozie.util.TestOozieRollingPolicy.testDeletingOldFiles(TestOozieRollingPolicy.java:47)
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3476) Migrate classes without setup/tearDown to JUnit4
[ https://issues.apache.org/jira/browse/OOZIE-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877787#comment-16877787 ] Julia Kinga Marton commented on OOZIE-3476: --- thank you [~asalamon74] for the contribution! +1, committed to master > Migrate classes without setup/tearDown to JUnit4 > > > Key: OOZIE-3476 > URL: https://issues.apache.org/jira/browse/OOZIE-3476 > Project: Oozie > Issue Type: Sub-task > Components: tests >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-3476-01.patch, OOZIE-3476-02.patch, > OOZIE-3476-03.patch > > > There are several classes where we really don't need to extend XTestCase or > even TestCase, we can easily convert them to JUnit4. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3476) Migrate classes without setup/tearDown to JUnit4
[ https://issues.apache.org/jira/browse/OOZIE-3476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3476: -- Summary: Migrate classes without setup/tearDown to JUnit4 (was: Migrate several classes to JUnit4) > Migrate classes without setup/tearDown to JUnit4 > > > Key: OOZIE-3476 > URL: https://issues.apache.org/jira/browse/OOZIE-3476 > Project: Oozie > Issue Type: Sub-task > Components: tests >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-3476-01.patch, OOZIE-3476-02.patch, > OOZIE-3476-03.patch > > > There are several classes where we really don't need to extend XTestCase or > even TestCase, we can easily convert them to JUnit4. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877756#comment-16877756 ] Julia Kinga Marton commented on OOZIE-3522: --- Thank you [~asalamon74] for the contribution! +1, committed to master > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3266) Coord action rerun support RERUN_SKIP_NODES option
[ https://issues.apache.org/jira/browse/OOZIE-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16877740#comment-16877740 ] Julia Kinga Marton commented on OOZIE-3266: --- [~zuston], please upload the new patch here as well, to trigger the precommit job. Also please cover the new feature with test cases and update the documentation as well (DG_CoordinatorRerun.md) > Coord action rerun support RERUN_SKIP_NODES option > -- > > Key: OOZIE-3266 > URL: https://issues.apache.org/jira/browse/OOZIE-3266 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.1.0 >Reporter: TIAN XING >Assignee: Junfan Zhang >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3266-v1.txt, OOZIE-3266-v2.txt > > > currently, when you re-run a workflow job, you have 3 options > # re-run all of its action nodes > # re-run failed nodes only > # re-run with specified nodes skipped > if this workflow job is generated by a coord action. you can re-run this > coord action with 2 options > # re-run all of the workflow action nodes (generate a new workflow job id) > # re-run failed workflow action nodes only (workflow job id not changed) > now we want to add a another option > - re-run with specified workflow nodes skipped (workflow job id not changed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[
https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16876045#comment-16876045
]
Julia Kinga Marton commented on OOZIE-3513:
---
Thank you @asalamon for the contribution! +1, committed to master
> Migrate from Preconditions.checkNotNull and ParamChecker.notNull
>
>
> Key: OOZIE-3513
> URL: https://issues.apache.org/jira/browse/OOZIE-3513
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch,
> OOZIE-3513-03.patch, OOZIE-3513-04.patch, OOZIE-3513-05.patch
>
>
> We currently use both Guava's {{Preconditions.checkNotNull}} and our own
> {{ParamChecker.notNull}} to check for null arguments. Instead we should use
> the standard {{Objects.requireNonNull}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3266) Coord action rerun support RERUN_SKIP_NODES option
[ https://issues.apache.org/jira/browse/OOZIE-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16875971#comment-16875971 ] Julia Kinga Marton commented on OOZIE-3266: --- Thank you [~zuston]! I have left some comments on the RB. > Coord action rerun support RERUN_SKIP_NODES option > -- > > Key: OOZIE-3266 > URL: https://issues.apache.org/jira/browse/OOZIE-3266 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.1.0 >Reporter: TIAN XING >Assignee: Junfan Zhang >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3266-v1.txt, OOZIE-3266-v2.txt > > > currently, when you re-run a workflow job, you have 3 options > # re-run all of its action nodes > # re-run failed nodes only > # re-run with specified nodes skipped > if this workflow job is generated by a coord action. you can re-run this > coord action with 2 options > # re-run all of the workflow action nodes (generate a new workflow job id) > # re-run failed workflow action nodes only (workflow job id not changed) > now we want to add a another option > - re-run with specified workflow nodes skipped (workflow job id not changed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2907) Delete PrepareActionsDriver from oozie-sharelib
[ https://issues.apache.org/jira/browse/OOZIE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874734#comment-16874734 ] Julia Kinga Marton commented on OOZIE-2907: --- Thank you [~asalamon74] for the contribution! +1, committed to master > Delete PrepareActionsDriver from oozie-sharelib > --- > > Key: OOZIE-2907 > URL: https://issues.apache.org/jira/browse/OOZIE-2907 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 5.0.0 >Reporter: Peter Bacsko >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-2907-01.patch, OOZIE-2907-02.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2422) Recovery service loads jobs which doesn't need recovery
[
https://issues.apache.org/jira/browse/OOZIE-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16874160#comment-16874160
]
Julia Kinga Marton commented on OOZIE-2422:
---
[~satishsaley] can you please rebase your patch on top the actual master?
> Recovery service loads jobs which doesn't need recovery
> ---
>
> Key: OOZIE-2422
> URL: https://issues.apache.org/jira/browse/OOZIE-2422
> Project: Oozie
> Issue Type: Bug
>Reporter: Purshotam Shah
>Assignee: Satish Subhashrao Saley
>Priority: Major
> Attachments: OOZIE-2422-1.patch
>
>
> {code}
> @NamedQuery(name = "GET_COORD_ACTIONS_FOR_RECOVERY_OLDER_THAN", query
> = "select a.id, a.jobId, a.statusStr, a.externalId, a.pending from
> CoordinatorActionBean a where a.pending > 0 AND (a.statusStr = 'SUSPENDED' OR
> a.statusStr = 'KILLED' OR a.statusStr = 'RUNNING') AND
> a.lastModifiedTimestamp <= :lastModifiedTime"),
> {code}
> Recovery service use above sql to recover killed/suspended/running action and
> in code it checks for external id. Checking of externalId can be done in sql
> itself.
> {code}
> else if (caction.getStatus() == CoordinatorActionBean.Status.SUSPENDED) {
> if (caction.getExternalId() != null &&
> caction.getPending() > 1) {
> queueCallable(new
> SuspendXCommand(caction.getExternalId()));
> log.debug("Recover a SUSPENDED coord action
> and resubmit SuspendXCommand :"
> + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.KILLED) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> KillXCommand(caction.getExternalId()));
> log.debug("Recover a KILLED coord action and
> resubmit KillXCommand :" + caction.getId());
> }
> }
> else if (caction.getStatus() ==
> CoordinatorActionBean.Status.RUNNING) {
> if (caction.getExternalId() != null) {
> queueCallable(new
> ResumeXCommand(caction.getExternalId()));
> log.debug("Recover a RUNNING coord action and
> resubmit ResumeXCommand :" + caction.getId());
> }
> }
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[
https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16873989#comment-16873989
]
Julia Kinga Marton commented on OOZIE-3513:
---
[~asalamon74] can you please upload your patch to ReviewBoard?
> Migrate from Preconditions.checkNotNull and ParamChecker.notNull
>
>
> Key: OOZIE-3513
> URL: https://issues.apache.org/jira/browse/OOZIE-3513
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch,
> OOZIE-3513-03.patch, OOZIE-3513-04.patch
>
>
> We currently use both Guava's {{Preconditions.checkNotNull}} and our own
> {{ParamChecker.notNull}} to check for null arguments. Instead we should use
> the standard {{Objects.requireNonNull}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3516) Misleading warning: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: yarnRM
[
https://issues.apache.org/jira/browse/OOZIE-3516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16868464#comment-16868464
]
Julia Kinga Marton commented on OOZIE-3516:
---
[~asalamon74],
The log message is the following:
{code:java}
2019-06-20 13:42:51,486 WARN [main] hadoop.HadoopTokenHelper
(XLog.java:log(523)) - An error happened while trying to get server principal.
Getting it from service principal anyway.
2019-06-20 13:42:51,487 TRACE [main] hadoop.HadoopTokenHelper
(XLog.java:log(520)) - java.lang.IllegalArgumentException: Does not contain a
valid host:port authority: rm-ha-uri
java.lang.IllegalArgumentException: Does not contain a valid host:port
authority: rm-ha-uri
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:212)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
at
org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:73)
at
org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:51)
at
org.apache.oozie.action.hadoop.TestHadoopTokenHelper.testGetMRDelegationTokenRenewer(TestHadoopTokenHelper.java:42){code}
> Misleading warning: java.lang.IllegalArgumentException: Does not contain a
> valid host:port authority: yarnRM
>
>
> Key: OOZIE-3516
> URL: https://issues.apache.org/jira/browse/OOZIE-3516
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3516-001.patch
>
>
> Starting from 5.0.0 there is a misleading warning message in the Oozie logs:
> {code}
> 2019-06-17 05:30:19,335 WARN
> org.apache.oozie.action.hadoop.HadoopTokenHelper:
> SERVER[nightly6x-1.vpc.cloudera.com] USER[admin] GROUP[-] TOKEN[] APP[Batch
> job for PySpark Pi Estimator Job] JOB[001-190617003335655-oozie-oozi-W]
> ACTION[001-190617003335655-oozie-oozi-W@spark2-3833] An error happened
> while trying to get server principal. Getting it from service principal
> anyway.
> java.lang.IllegalArgumentException: Does not contain a valid host:port
> authority: yarnRM
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:213)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
> at
> org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:73)
> at
> org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:51)
> at
> org.apache.oozie.action.hadoop.YarnRMCredentials.updateCredentials(YarnRMCredentials.java:55)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1503)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1053)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1601)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:243)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:68)
> at org.apache.oozie.command.XCommand.call(XCommand.java:291)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:363)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> This warning is harmless, but it can case misunderstandings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[
https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867450#comment-16867450
]
Julia Kinga Marton commented on OOZIE-3513:
---
[~asalamon74] please check the long lines.
> Migrate from Preconditions.checkNotNull and ParamChecker.notNull
>
>
> Key: OOZIE-3513
> URL: https://issues.apache.org/jira/browse/OOZIE-3513
> Project: Oozie
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch
>
>
> We currently use both Guava's {{Preconditions.checkNotNull}} and our own
> {{ParamChecker.notNull}} to check for null arguments. Instead we should use
> the standard {{Objects.requireNonNull}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3515) Upgrade sqoop version to 1.4.7
[ https://issues.apache.org/jira/browse/OOZIE-3515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867448#comment-16867448 ] Julia Kinga Marton commented on OOZIE-3515: --- Thank you [~asalamon74] for the contribution! +1, committed to master > Upgrade sqoop version to 1.4.7 > -- > > Key: OOZIE-3515 > URL: https://issues.apache.org/jira/browse/OOZIE-3515 > Project: Oozie > Issue Type: Bug >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3515-01.patch, OOZIE-3515-02.patch > > > Oozie still uses Sqoop 1.4.3 which was released in 2013, it's time to upgrade > to the latest sqoop (1.4.7) which was released in 2018. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3266) Coord action rerun support RERUN_SKIP_NODES option
[ https://issues.apache.org/jira/browse/OOZIE-3266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16867415#comment-16867415 ] Julia Kinga Marton commented on OOZIE-3266: --- [~zuston], can you please upload your patch to ReviewBoard? > Coord action rerun support RERUN_SKIP_NODES option > -- > > Key: OOZIE-3266 > URL: https://issues.apache.org/jira/browse/OOZIE-3266 > Project: Oozie > Issue Type: New Feature >Affects Versions: 5.1.0 >Reporter: TIAN XING >Assignee: Junfan Zhang >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3266-v1.txt, OOZIE-3266-v2.txt > > > currently, when you re-run a workflow job, you have 3 options > # re-run all of its action nodes > # re-run failed nodes only > # re-run with specified nodes skipped > if this workflow job is generated by a coord action. you can re-run this > coord action with 2 options > # re-run all of the workflow action nodes (generate a new workflow job id) > # re-run failed workflow action nodes only (workflow job id not changed) > now we want to add a another option > - re-run with specified workflow nodes skipped (workflow job id not changed) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3516) Misleading warning: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: yarnRM
[
https://issues.apache.org/jira/browse/OOZIE-3516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3516:
--
Attachment: OOZIE-3516-001.patch
> Misleading warning: java.lang.IllegalArgumentException: Does not contain a
> valid host:port authority: yarnRM
>
>
> Key: OOZIE-3516
> URL: https://issues.apache.org/jira/browse/OOZIE-3516
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.0.0
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3516-001.patch
>
>
> Starting from 5.0.0 there is a misleading warning message in the Oozie logs:
> {code}
> 2019-06-17 05:30:19,335 WARN
> org.apache.oozie.action.hadoop.HadoopTokenHelper:
> SERVER[nightly6x-1.vpc.cloudera.com] USER[admin] GROUP[-] TOKEN[] APP[Batch
> job for PySpark Pi Estimator Job] JOB[001-190617003335655-oozie-oozi-W]
> ACTION[001-190617003335655-oozie-oozi-W@spark2-3833] An error happened
> while trying to get server principal. Getting it from service principal
> anyway.
> java.lang.IllegalArgumentException: Does not contain a valid host:port
> authority: yarnRM
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:213)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
> at
> org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:73)
> at
> org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:51)
> at
> org.apache.oozie.action.hadoop.YarnRMCredentials.updateCredentials(YarnRMCredentials.java:55)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1503)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1053)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1601)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:243)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:68)
> at org.apache.oozie.command.XCommand.call(XCommand.java:291)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:363)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
> This warning is harmless, but it can case misunderstandings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3516) Misleading warning: java.lang.IllegalArgumentException: Does not contain a valid host:port authority: yarnRM
Julia Kinga Marton created OOZIE-3516:
-
Summary: Misleading warning: java.lang.IllegalArgumentException:
Does not contain a valid host:port authority: yarnRM
Key: OOZIE-3516
URL: https://issues.apache.org/jira/browse/OOZIE-3516
Project: Oozie
Issue Type: Bug
Affects Versions: 5.0.0
Reporter: Julia Kinga Marton
Assignee: Julia Kinga Marton
Starting from 5.0.0 there is a misleading warning message in the Oozie logs:
{code}
2019-06-17 05:30:19,335 WARN org.apache.oozie.action.hadoop.HadoopTokenHelper:
SERVER[nightly6x-1.vpc.cloudera.com] USER[admin] GROUP[-] TOKEN[] APP[Batch job
for PySpark Pi Estimator Job] JOB[001-190617003335655-oozie-oozi-W]
ACTION[001-190617003335655-oozie-oozi-W@spark2-3833] An error happened
while trying to get server principal. Getting it from service principal anyway.
java.lang.IllegalArgumentException: Does not contain a valid host:port
authority: yarnRM
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:213)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:153)
at
org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:73)
at
org.apache.oozie.action.hadoop.HadoopTokenHelper.getServerPrincipal(HadoopTokenHelper.java:51)
at
org.apache.oozie.action.hadoop.YarnRMCredentials.updateCredentials(YarnRMCredentials.java:55)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.setCredentialTokens(JavaActionExecutor.java:1503)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1053)
at
org.apache.oozie.action.hadoop.JavaActionExecutor.start(JavaActionExecutor.java:1601)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:243)
at
org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:68)
at org.apache.oozie.command.XCommand.call(XCommand.java:291)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:363)
at
org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:210)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
{code}
This warning is harmless, but it can case misunderstandings.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3507) Upgrade to Dozer 6
[ https://issues.apache.org/jira/browse/OOZIE-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860798#comment-16860798 ] Julia Kinga Marton commented on OOZIE-3507: --- Thank you [~asalamon74] for the contribution! +1, committed to master. > Upgrade to Dozer 6 > -- > > Key: OOZIE-3507 > URL: https://issues.apache.org/jira/browse/OOZIE-3507 > Project: Oozie > Issue Type: Bug >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3507-01-wip.patch > > > Oozie uses [Dozer|http://dozer.sourceforge.net/] library. We use the [latest > version|https://mvnrepository.com/artifact/net.sf.dozer/dozer] (5.5.1) which > was released in 2014. > The project has been revived, which should check the new 6.x version ( > https://github.com/DozerMapper/dozer ) and switch to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3492) [spark-action] Missing HADOOP_CONF_DIR property
[
https://issues.apache.org/jira/browse/OOZIE-3492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860634#comment-16860634
]
Julia Kinga Marton commented on OOZIE-3492:
---
Thank you [~asalamon74] for the contribution! Is not the nicest one, but I
don't have any other better ideas. +1, committed to master
> [spark-action] Missing HADOOP_CONF_DIR property
> ---
>
> Key: OOZIE-3492
> URL: https://issues.apache.org/jira/browse/OOZIE-3492
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3492-001.patch
>
>
> Spark action fails with the following error:
> {noformat}
> Failing Oozie Launcher, When running with master 'yarn-cluster' either
> HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.
> org.apache.spark.SparkException: When running with master 'yarn-cluster'
> either HADOOP_CONF_DIR or YARN_CONF_DIR must be set in the environment.
> at
> org.apache.spark.deploy.SparkSubmitArguments.error(SparkSubmitArguments.scala:657)
> at
> org.apache.spark.deploy.SparkSubmitArguments.validateSubmitArguments(SparkSubmitArguments.scala:290)
> at
> org.apache.spark.deploy.SparkSubmitArguments.validateArguments(SparkSubmitArguments.scala:251)
> at
> org.apache.spark.deploy.SparkSubmitArguments.(SparkSubmitArguments.scala:120)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2$$anon$1.(SparkSubmit.scala:926)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.parseArguments(SparkSubmit.scala:926)
> at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:81)
> at
> org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:939)
> at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:948)
> at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
> at
> org.apache.oozie.action.hadoop.SparkMain.runSpark(SparkMain.java:186)
> at org.apache.oozie.action.hadoop.SparkMain.run(SparkMain.java:93)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:104)
> at org.apache.oozie.action.hadoop.SparkMain.main(SparkMain.java:60)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:410)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$300(LauncherAM.java:55)
> at
> org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:223)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:217)
> at
> org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:153)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
> at
> org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:141){noformat}
> if the followings are true:
> * yarn.nodemanager.env-whitelist does not contain {{HADOOP_CONF_DIR}}
> * Hadoop version >= 3.1 (or YARN-7677 is backported)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3495) Upgrade hive version to 1.2.2
[ https://issues.apache.org/jira/browse/OOZIE-3495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16860578#comment-16860578 ] Julia Kinga Marton commented on OOZIE-3495: --- Thank you [~nobigo] for the contribution! +1, committed to master > Upgrade hive version to 1.2.2 > - > > Key: OOZIE-3495 > URL: https://issues.apache.org/jira/browse/OOZIE-3495 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon >Assignee: duan xiong >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3495-001.patch, OOZIE-3495-002.patch > > > Oozie still uses hive 1.2.0 which was released in 2015. Upgrading to 2.x (or > 3.x) is probably not a good idea in an Oozie minor release, but we could > safely upgrade it to the latest 1.2.x version which is 1.2.2 (released in > 2017). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2907) Delete PrepareActionsDriver from oozie-sharelib
[ https://issues.apache.org/jira/browse/OOZIE-2907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858587#comment-16858587 ] Julia Kinga Marton commented on OOZIE-2907: --- [~asalamon74] why is TestPrepareActionsDriver kept if PrepareActionsDriver is deleted? > Delete PrepareActionsDriver from oozie-sharelib > --- > > Key: OOZIE-2907 > URL: https://issues.apache.org/jira/browse/OOZIE-2907 > Project: Oozie > Issue Type: Sub-task >Affects Versions: 5.0.0 >Reporter: Peter Bacsko >Assignee: Andras Salamon >Priority: Minor > Attachments: OOZIE-2907-01.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-2779) Mask Hive2 action Beeline JDBC password
[
https://issues.apache.org/jira/browse/OOZIE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858582#comment-16858582
]
Julia Kinga Marton commented on OOZIE-2779:
---
[~abhishekbafna] can you pleae rebase your patch on top the actual master and
consider [~asalamon74]'s suggestion?
> Mask Hive2 action Beeline JDBC password
> ---
>
> Key: OOZIE-2779
> URL: https://issues.apache.org/jira/browse/OOZIE-2779
> Project: Oozie
> Issue Type: Bug
> Components: action
>Reporter: Abhishek Bafna
>Assignee: Abhishek Bafna
>Priority: Major
> Labels: security
> Fix For: trunk
>
> Attachments: OOZIE-2779-00.patch, OOZIE-2779-01.patch,
> OOZIE-2779-01.patch
>
>
> Hive2 Oozie launcher job prints the JDBC password into launcher stdout logs.
> {noformat}
> Beeline command arguments :
> -u
> jdbc:hive2://source-1:1/default
> -n
> ambari-qa
> -p
> DUMMY
> -d
> org.apache.hive.jdbc.HiveDriver
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-2879) Remove unused class SLAStore and related classes
[
https://issues.apache.org/jira/browse/OOZIE-2879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858579#comment-16858579
]
Julia Kinga Marton commented on OOZIE-2879:
---
thank you [~asalamon74] for the contribution! +1, committed to master
> Remove unused class SLAStore and related classes
>
>
> Key: OOZIE-2879
> URL: https://issues.apache.org/jira/browse/OOZIE-2879
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Peter Bacsko
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-2879-01.patch
>
>
> While working on OOZIE-2854, I was looking for classes which use
> {{EntityManager}}. There is a class called {{SLAStore}} which uses it, but
> retrieving the call hierarchy revealed that only a test class invokes the
> methods of {{SLAStore}}.
> This class has been unused for a long time - it was deprecated in OOZIE-914
> (see commit:
> https://github.com/apache/oozie/commit/c95c809a55005b44b05f0c496e4a3fd75a26be91),
> so we should remove it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3505) [Java 11] Fix TestDBLoadDump
[
https://issues.apache.org/jira/browse/OOZIE-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3505:
--
Attachment: OOZIE-3505-003.patch
> [Java 11] Fix TestDBLoadDump
>
>
> Key: OOZIE-3505
> URL: https://issues.apache.org/jira/browse/OOZIE-3505
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3505-001.patch, OOZIE-3505-002.patch,
> OOZIE-3505-003.patch
>
>
> There are a couple of test cases in TestDBLoadDump which are failing with the
> following exception with JDK11:
> {code:bash}
> org.apache.oozie.tools.TestDBLoadDump$ExitException
> at
> org.apache.oozie.tools.TestDBLoadDump$NoExitSecurityManager.checkExit(TestDBLoadDump.java:285)
> at java.base/java.lang.Runtime.exit(Runtime.java:113)
> at java.base/java.lang.System.exit(System.java:1746)
> at org.apache.oozie.tools.OozieDBImportCLI.main(OozieDBImportCLI.java:158)
> at org.apache.oozie.tools.TestDBLoadDump.importToDB(TestDBLoadDump.java:219)
> at
> org.apache.oozie.tools.TestDBLoadDump.importValidDataToDB(TestDBLoadDump.java:211)
> at
> org.apache.oozie.tools.TestDBLoadDump.testImportToNonExistingTablesSucceedsOnHsqldb(TestDBLoadDump.java:133)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at junit.framework.TestCase.runTest(TestCase.java:176){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3505) [Java 11] Fix TestDBLoadDump
[
https://issues.apache.org/jira/browse/OOZIE-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3505:
--
Attachment: OOZIE-3505-002.patch
> [Java 11] Fix TestDBLoadDump
>
>
> Key: OOZIE-3505
> URL: https://issues.apache.org/jira/browse/OOZIE-3505
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3505-001.patch, OOZIE-3505-002.patch
>
>
> There are a couple of test cases in TestDBLoadDump which are failing with the
> following exception with JDK11:
> {code:bash}
> org.apache.oozie.tools.TestDBLoadDump$ExitException
> at
> org.apache.oozie.tools.TestDBLoadDump$NoExitSecurityManager.checkExit(TestDBLoadDump.java:285)
> at java.base/java.lang.Runtime.exit(Runtime.java:113)
> at java.base/java.lang.System.exit(System.java:1746)
> at org.apache.oozie.tools.OozieDBImportCLI.main(OozieDBImportCLI.java:158)
> at org.apache.oozie.tools.TestDBLoadDump.importToDB(TestDBLoadDump.java:219)
> at
> org.apache.oozie.tools.TestDBLoadDump.importValidDataToDB(TestDBLoadDump.java:211)
> at
> org.apache.oozie.tools.TestDBLoadDump.testImportToNonExistingTablesSucceedsOnHsqldb(TestDBLoadDump.java:133)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at junit.framework.TestCase.runTest(TestCase.java:176){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3505) [Java 11] Fix TestDBLoadDump
[
https://issues.apache.org/jira/browse/OOZIE-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16858410#comment-16858410
]
Julia Kinga Marton commented on OOZIE-3505:
---
Starting from Java9, the default locale handling has changed:
[https://bugs.openjdk.java.net/browse/JDK-8206961].
Gson uses the default US locale when it writes out data:
[https://github.com/google/gson/blob/master/gson/src/main/java/com/google/gson/internal/bind/DateTypeAdapter.java#L96]
[https://github.com/google/gson/blob/master/gson/src/main/java/com/google/gson/internal/bind/DateTypeAdapter.java#L61]
With JDK 11
{code:java}
DateFormat.getDateTimeInstance(DateFormat.DEFAULT, DateFormat.DEFAULT,
Locale.US)
{code}
will return the following format "MMM d, y, h:mm:ss a" instead of "MMM, d
h:mm:ss a", returned until now in JDK 8.
> [Java 11] Fix TestDBLoadDump
>
>
> Key: OOZIE-3505
> URL: https://issues.apache.org/jira/browse/OOZIE-3505
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3505-001.patch
>
>
> There are a couple of test cases in TestDBLoadDump which are failing with the
> following exception with JDK11:
> {code:bash}
> org.apache.oozie.tools.TestDBLoadDump$ExitException
> at
> org.apache.oozie.tools.TestDBLoadDump$NoExitSecurityManager.checkExit(TestDBLoadDump.java:285)
> at java.base/java.lang.Runtime.exit(Runtime.java:113)
> at java.base/java.lang.System.exit(System.java:1746)
> at org.apache.oozie.tools.OozieDBImportCLI.main(OozieDBImportCLI.java:158)
> at org.apache.oozie.tools.TestDBLoadDump.importToDB(TestDBLoadDump.java:219)
> at
> org.apache.oozie.tools.TestDBLoadDump.importValidDataToDB(TestDBLoadDump.java:211)
> at
> org.apache.oozie.tools.TestDBLoadDump.testImportToNonExistingTablesSucceedsOnHsqldb(TestDBLoadDump.java:133)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at junit.framework.TestCase.runTest(TestCase.java:176){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3507) Upgrade to Dozer 6
[ https://issues.apache.org/jira/browse/OOZIE-3507?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856633#comment-16856633 ] Julia Kinga Marton commented on OOZIE-3507: --- The patch does compile with JDK11 as well. > Upgrade to Dozer 6 > -- > > Key: OOZIE-3507 > URL: https://issues.apache.org/jira/browse/OOZIE-3507 > Project: Oozie > Issue Type: Bug >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3507-01-wip.patch > > > Oozie uses [Dozer|http://dozer.sourceforge.net/] library. We use the [latest > version|https://mvnrepository.com/artifact/net.sf.dozer/dozer] (5.5.1) which > was released in 2014. > The project has been revived, which should check the new 6.x version ( > https://github.com/DozerMapper/dozer ) and switch to it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3510) TestProxyUserService.testInvalidGroup fails if executed by nobody
[
https://issues.apache.org/jira/browse/OOZIE-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856578#comment-16856578
]
Julia Kinga Marton commented on OOZIE-3510:
---
Thank you [~asalamon74] for the contribution! +1, committed to master
> TestProxyUserService.testInvalidGroup fails if executed by nobody
> -
>
> Key: OOZIE-3510
> URL: https://issues.apache.org/jira/browse/OOZIE-3510
> Project: Oozie
> Issue Type: Bug
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3510-01.patch
>
>
> TestProxyUserService.testInvalidGroup fails if executed by nobody with the
> following error message:
> {noformat}junit.framework.AssertionFailedError
> at junit.framework.Assert.fail(Assert.java:55)
> at junit.framework.Assert.fail(Assert.java:64)
> at junit.framework.TestCase.fail(TestCase.java:235)
> at
> org.apache.oozie.service.TestProxyUserService.testInvalidGroup(TestProxyUserService.java:261)
> ...
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3505) [Java 11] Fix TestDBLoadDump
[
https://issues.apache.org/jira/browse/OOZIE-3505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3505:
--
Attachment: OOZIE-3505-001.patch
> [Java 11] Fix TestDBLoadDump
>
>
> Key: OOZIE-3505
> URL: https://issues.apache.org/jira/browse/OOZIE-3505
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3505-001.patch
>
>
> There are a couple of test cases in TestDBLoadDump which are failing with the
> following exception with JDK11:
> {code:bash}
> org.apache.oozie.tools.TestDBLoadDump$ExitException
> at
> org.apache.oozie.tools.TestDBLoadDump$NoExitSecurityManager.checkExit(TestDBLoadDump.java:285)
> at java.base/java.lang.Runtime.exit(Runtime.java:113)
> at java.base/java.lang.System.exit(System.java:1746)
> at org.apache.oozie.tools.OozieDBImportCLI.main(OozieDBImportCLI.java:158)
> at org.apache.oozie.tools.TestDBLoadDump.importToDB(TestDBLoadDump.java:219)
> at
> org.apache.oozie.tools.TestDBLoadDump.importValidDataToDB(TestDBLoadDump.java:211)
> at
> org.apache.oozie.tools.TestDBLoadDump.testImportToNonExistingTablesSucceedsOnHsqldb(TestDBLoadDump.java:133)
> at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:566)
> at junit.framework.TestCase.runTest(TestCase.java:176){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3511) Different services orders defined in oozie.services.ext can lead to different results
Julia Kinga Marton created OOZIE-3511:
-
Summary: Different services orders defined in oozie.services.ext
can lead to different results
Key: OOZIE-3511
URL: https://issues.apache.org/jira/browse/OOZIE-3511
Project: Oozie
Issue Type: Bug
Reporter: Julia Kinga Marton
Based on the order we define the services using {{oozie.services.ext}}
property can lead to different results.
For example in the following case the JMS services will fail to start correctly:
{noformat}
org.apache.oozie.service.ZKLocksService,org.apache.oozie.service.ZKXLogStreamingService,org.apache.oozie.service.ZKJobsConcurrencyService,org.apache.oozie.service.ZKUUIDService,org.apache.oozie.service.EventHandlerService,org.apache.oozie.sla.service.SLAService,org.apache.oozie.service.JMSAccessorService,org.apache.oozie.service.JMSTopicService,org.apache.oozie.service.PartitionDependencyManagerService,org.apache.oozie.service.HCatAccessorService,org.apache.oozie.service.MetricsInstrumentationService
{noformat}
If we exchange the JMSTopicService with the EventHandlerService, than if will
work as expected.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3499) [Java 11] Fix TestLiteWorkflowAppParser
[
https://issues.apache.org/jira/browse/OOZIE-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3499:
--
Attachment: OOZIE-3499-003.patch
> [Java 11] Fix TestLiteWorkflowAppParser
> ---
>
> Key: OOZIE-3499
> URL: https://issues.apache.org/jira/browse/OOZIE-3499
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3499-001.patch, OOZIE-3499-002.patch,
> OOZIE-3499-003.patch
>
>
> All the tests from TestLiteWorkflowAppParser, where global configurations are
> used are failing because with Java11 the order of the properties is not the
> same as with Java8.
> {code:xml}
>
> ${foo}
> bar
>
>
> a
> A
>
>
> b
> B
>
>
> {code}
> With Java 11 will have the same order as in the xml, but with Java 8 the
> order is changed, and we are asserting to the "wrong" order.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3499) [Java 11] Fix TestLiteWorkflowAppParser
[
https://issues.apache.org/jira/browse/OOZIE-3499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3499:
--
Attachment: OOZIE-3499-002.patch
> [Java 11] Fix TestLiteWorkflowAppParser
> ---
>
> Key: OOZIE-3499
> URL: https://issues.apache.org/jira/browse/OOZIE-3499
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3499-001.patch, OOZIE-3499-002.patch
>
>
> All the tests from TestLiteWorkflowAppParser, where global configurations are
> used are failing because with Java11 the order of the properties is not the
> same as with Java8.
> {code:xml}
>
> ${foo}
> bar
>
>
> a
> A
>
>
> b
> B
>
>
> {code}
> With Java 11 will have the same order as in the xml, but with Java 8 the
> order is changed, and we are asserting to the "wrong" order.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3509) [Java 11] Fix TestMapReduceActionExecutor test failures
Julia Kinga Marton created OOZIE-3509:
-
Summary: [Java 11] Fix TestMapReduceActionExecutor test failures
Key: OOZIE-3509
URL: https://issues.apache.org/jira/browse/OOZIE-3509
Project: Oozie
Issue Type: Sub-task
Reporter: Julia Kinga Marton
Assignee: Julia Kinga Marton
{{TestMapReduceActionExecutor#testJobNameSetForMapReduceChild}} and
{{TestMapReduceActionExecutor#testMapReduceWithUberJarEnabled}} are failing:
{code:bash}
TestMapReduceActionExecutor.testJobNameSetForMapReduceChild:801->_testSubmit:423
expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>
[ERROR]
TestMapReduceActionExecutor.testMapReduceWithUberJarEnabled:820->_testMapReduceWithUberJar:736->_testSubmit:423
expected:<[SUCCEED]ED> but was:<[FAILED/KILL]ED>
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3508) [Java 11] Fix TestHive2ActionExecutor#testHive2Action
Julia Kinga Marton created OOZIE-3508:
-
Summary: [Java 11] Fix TestHive2ActionExecutor#testHive2Action
Key: OOZIE-3508
URL: https://issues.apache.org/jira/browse/OOZIE-3508
Project: Oozie
Issue Type: Sub-task
Reporter: Julia Kinga Marton
TestHive2ActionExecutor#testHive2Action if failing with the following error
message:
{code:bash}
java.lang.RuntimeException: Error applying authorization policy on hive
configuration: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hive.service.cli.CLIService.init(CLIService.java:114)
at org.apache.hive.service.CompositeService.init(CompositeService.java:59)
at org.apache.hive.service.server.HiveServer2.init(HiveServer2.java:100)
at org.apache.oozie.test.hive.MiniHS2.start(MiniHS2.java:101)
at org.apache.oozie.test.XTestCase.setupHiveServer2(XTestCase.java:1147)
at
org.apache.oozie.action.hadoop.TestHive2ActionExecutor.testHive2Action(TestHive2ActionExecutor.java:188)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at junit.framework.TestCase.runTest(TestCase.java:176)
at junit.framework.TestCase.runBare(TestCase.java:141)
at junit.framework.TestResult$1.protect(TestResult.java:122)
at junit.framework.TestResult.runProtected(TestResult.java:142)
at junit.framework.TestResult.run(TestResult.java:125)
at junit.framework.TestCase.run(TestCase.java:129)
at junit.framework.TestSuite.runTest(TestSuite.java:255)
at junit.framework.TestSuite.run(TestSuite.java:250)
at
org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84)
at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
at
com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
at
com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
at
com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: Unable to
instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:519)
at
org.apache.hive.service.cli.CLIService.applyAuthorizationConfigPolicy(CLIService.java:127)
at org.apache.hive.service.cli.CLIService.init(CLIService.java:112)
... 23 more
Caused by: java.lang.RuntimeException: Unable to instantiate
org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
at
org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
at
org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:500)
... 25 more
Caused by: java.lang.reflect.InvocationTargetException
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
at
org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
... 31 more
Caused by: javax.jdo.JDOFatalInternalException: The java type java.lang.Long
(jdbc-type="", sql-type="") cant be mapped for this datastore. No mapping is
available.
NestedThrowables:
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3505) [Java 11] Fix TestDBLoadDump
Julia Kinga Marton created OOZIE-3505:
-
Summary: [Java 11] Fix TestDBLoadDump
Key: OOZIE-3505
URL: https://issues.apache.org/jira/browse/OOZIE-3505
Project: Oozie
Issue Type: Sub-task
Reporter: Julia Kinga Marton
Assignee: Julia Kinga Marton
There are a couple of test cases in TestDBLoadDump which are failing with the
following exception with JDK11:
{code:bash}
org.apache.oozie.tools.TestDBLoadDump$ExitException
at
org.apache.oozie.tools.TestDBLoadDump$NoExitSecurityManager.checkExit(TestDBLoadDump.java:285)
at java.base/java.lang.Runtime.exit(Runtime.java:113)
at java.base/java.lang.System.exit(System.java:1746)
at org.apache.oozie.tools.OozieDBImportCLI.main(OozieDBImportCLI.java:158)
at org.apache.oozie.tools.TestDBLoadDump.importToDB(TestDBLoadDump.java:219)
at
org.apache.oozie.tools.TestDBLoadDump.importValidDataToDB(TestDBLoadDump.java:211)
at
org.apache.oozie.tools.TestDBLoadDump.testImportToNonExistingTablesSucceedsOnHsqldb(TestDBLoadDump.java:133)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native
Method)
at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at junit.framework.TestCase.runTest(TestCase.java:176){code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3503) TestJavaActionExecutor.testSubmitWithLauncherQueue unit test failure
[
https://issues.apache.org/jira/browse/OOZIE-3503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16851839#comment-16851839
]
Julia Kinga Marton commented on OOZIE-3503:
---
Thank you [~asalamon74] for fixing this! +1, committed to master.
> TestJavaActionExecutor.testSubmitWithLauncherQueue unit test failure
>
>
> Key: OOZIE-3503
> URL: https://issues.apache.org/jira/browse/OOZIE-3503
> Project: Oozie
> Issue Type: Bug
> Components: tests
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3503-01.patch
>
>
> {{TestJavaActionExecutor.testSubmitWithLauncherQueue}} unit test fails if
> CapacityScheduler is strict and does not allow to use unknown queues. It
> gives the following error:
> {noformat}org.apache.oozie.action.ActionExecutorException: YarnException:
> Failed to submit application_1559057153957_0057 to YARN : Application
> application_1559057153957_0057 submitted by user test to unknown queue: test
> at
> org.apache.oozie.action.ActionExecutor.convertException(ActionExecutor.java:452)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1103)
> at
> org.apache.oozie.action.hadoop.TestJavaActionExecutor.submitAction(TestJavaActionExecutor.java:375)
> at
> org.apache.oozie.action.hadoop.TestJavaActionExecutor.submitAction(TestJavaActionExecutor.java:387)
> at
> org.apache.oozie.action.hadoop.TestJavaActionExecutor.testSubmitWithLauncherQueue(TestJavaActionExecutor.java:2588)
> ...
> Caused by: org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit
> application_1559057153957_0057 to YARN : Application
> application_1559057153957_0057 submitted by user test to unknown queue: test
> at
> org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.submitApplication(YarnClientImpl.java:322)
> at
> org.apache.oozie.action.hadoop.JavaActionExecutor.submitLauncher(JavaActionExecutor.java:1091)
> ... 45 more
> {noformat}
> We should use a queue name which is listed in the
> {{yarn.scheduler.capacity.root.queues}} property (
> https://github.com/apache/oozie/blob/master/core/src/test/java/org/apache/oozie/test/XTestCase.java#L1109
> )
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3504) [Java 11] Fix tests using PowerMock
[ https://issues.apache.org/jira/browse/OOZIE-3504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Julia Kinga Marton updated OOZIE-3504: -- Description: With JDK 11 the following tests are failing with initialisation error: * TestScriptLanguageActionExecutor * TestHCatCredentials was: With JDK 11 TestScriptLanguageActionExecutor#multibyteInputsAreAcceptedInScripts is failing with the initialisation error. > [Java 11] Fix tests using PowerMock > --- > > Key: OOZIE-3504 > URL: https://issues.apache.org/jira/browse/OOZIE-3504 > Project: Oozie > Issue Type: Sub-task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Major > > With JDK 11 the following tests are failing with initialisation error: > * TestScriptLanguageActionExecutor > * TestHCatCredentials > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
