[jira] [Created] (OOZIE-3593) Add Kafka Action
David Mollitor created OOZIE-3593: - Summary: Add Kafka Action Key: OOZIE-3593 URL: https://issues.apache.org/jira/browse/OOZIE-3593 Project: Oozie Issue Type: New Feature Components: action Reporter: David Mollitor Create Oozie Kafka Action to add/delete topics dynamically -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055230#comment-17055230
]
Hadoop QA commented on OOZIE-3592:
--
Testing JIRA OOZIE-3592
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [7] new bugs found below threshold in total that
must be fixed.
.{color:green}+1{color} There are no new bugs found in [webapp].
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:orange}0{color} There are [1] new bugs found in [sharelib/oozie]
that would be nice to have fixed.
.You can find the SpotBugs diff here: sharelib/oozie/findbugs-new.html
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:red}-1{color} There are [7] new bugs found below threshold in
[core] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
.At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
.This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
.At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3210
.{color:orange}Tests failed at first run:{color}
TestBlockingInputStream#testFastWritingBlockingInputStream
TestBlockingInputStream#testLimitedWritingBlockingInputStream
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1315/
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
>Reporter: Mate Juhasz
>Assignee: Mate
Failed: OOZIE-3592 PreCommit Build #1315
Jira: https://issues.apache.org/jira/browse/OOZIE-3592 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1315/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.16 MB...] [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Running test-patch task MODERNIZER Running modernizer op=report [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-clean.txt [TRACE] Modernizer errors before patch: 52 [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-patch.txt [TRACE] Modernizer errors after patch: 52 Testing JIRA OOZIE-3592 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 +1 the patch adds/modifies 1 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [7] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [webapp]. +1 There are no new bugs found in [fluent-job/fluent-job-api]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/hive2]. 0 There are [1] new bugs found in [sharelib/oozie] that would be nice to have fixed. You can find the SpotBugs diff here: sharelib/oozie/findbugs-new.html +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [client]. -1 There are [7] new bugs found below threshold in [core] that must be fixed, listing only the first [5] ones. You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html The top [5] most important SpotBugs errors are: At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175] At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199] This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206] At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127] +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 3210 Tests failed at first run: TestBlockingInputStream#testFastWritingBlockingInputStream TestBlockingInputStream#testLimitedWritingBlockingInputStream For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch +1 MODERNIZER -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1315/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 44380 0 100 4438 0 3344 0:00:01 0:00:01 --:--:--
[jira] [Commented] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17055087#comment-17055087
]
Hadoop QA commented on OOZIE-3592:
--
PreCommit-OOZIE-Build started
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: (was: OOZIE-3592-v2.patch)
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: OOZIE-3592-v2.patch
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3592) Do not print misleading System.exit() exception for successful jobs
[
https://issues.apache.org/jira/browse/OOZIE-3592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz updated OOZIE-3592:
---
Attachment: OOZIE-3592-v2.patch
> Do not print misleading System.exit() exception for successful jobs
> ---
>
> Key: OOZIE-3592
> URL: https://issues.apache.org/jira/browse/OOZIE-3592
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.2.0
>Reporter: Mate Juhasz
>Assignee: Mate Juhasz
>Priority: Major
> Fix For: trunk
>
> Attachments: OOZIE-3592-v1.patch, OOZIE-3592-v2.patch
>
>
> The following exception is logged by Oozie due to System.exit() calls in
> application JVM-s even after successful workflow actions:
> {noformat}
> Hadoop Job IDs executed by Sqoop: job_1583161376812_0018
> java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.oozie.action.hadoop.LauncherAM.runActionMain(LauncherAM.java:413)
> at
> org.apache.oozie.action.hadoop.LauncherAM.access$400(LauncherAM.java:55)
> at org.apache.oozie.action.hadoop.LauncherAM$2.run(LauncherAM.java:226)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.run(LauncherAM.java:220)
> at org.apache.oozie.action.hadoop.LauncherAM$1.run(LauncherAM.java:156)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.oozie.action.hadoop.LauncherAM.main(LauncherAM.java:144)
> Caused by: java.lang.SecurityException: Intercepted System.exit(0)
> at
> org.apache.oozie.action.hadoop.security.LauncherSecurityManager.checkExit(LauncherSecurityManager.java:57)
> at java.lang.Runtime.exit(Runtime.java:107)
> at java.lang.System.exit(System.java:971)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:260)
> at
> org.apache.oozie.action.hadoop.SqoopMain.runSqoopJob(SqoopMain.java:221)
> at org.apache.oozie.action.hadoop.SqoopMain.run(SqoopMain.java:206)
> at
> org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:107)
> at org.apache.oozie.action.hadoop.SqoopMain.main(SqoopMain.java:52)
> ... 16 more
> Intercepting System.exit(0)
> {noformat}
> Typically happens in Sqoop actions. This can be very misleading for users as
> they may suspect that some error has occurred even if their yarn application
> and Oozie workflow has been marked as succeeded.
> The "error" is already ignored by Oozie when the System.exit code is 0, but
> we should not print either this exception in such cases.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3586) Oozie spark actions using --keytab fail due to duplicate dist. cache
[
https://issues.apache.org/jira/browse/OOZIE-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054931#comment-17054931
]
Hadoop QA commented on OOZIE-3586:
--
Testing JIRA OOZIE-3586
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 1445
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:green}*+1 Overall result, good!, no -1s*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1314/
> Oozie spark actions using --keytab fail due to duplicate dist. cache
>
>
> Key: OOZIE-3586
> URL: https://issues.apache.org/jira/browse/OOZIE-3586
> Project: Oozie
> Issue Type: Bug
>Reporter: Janos Makai
>Assignee: Janos Makai
>Priority: Major
> Attachments: OOZIE-3586-001.patch, OOZIE-3586-002.patch,
> OOZIE-3586-003.patch
>
>
> In CDH 6.x, on the Hadoop 3 rebase, it is now a failure if items are added
> multiple times to the distributed cache. In CDH5, this was a warning. This is
> not an issue for most uses, as adding multiple times typically is user error,
> but this completely breaks Spark actions with keytabs (--keytab).
> Oozie spark actions add everything in the distributed cache of the launcher
> job to the distributed cache of the spark job, meaning the keytab is already
> there, then the --keytab argument tries to add it again causing the failure
> [0].
> [0]
> Caused by: java.lang.IllegalArgumentException: Attempt to add
> (.../filename.keytab#filename.keytab) multiple times to the distributed cache.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Failed: OOZIE-3586 PreCommit Build #1314
Jira: https://issues.apache.org/jira/browse/OOZIE-3586 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1314/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.10 MB...] [DEBUG] There are no new bugs found in [tools]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [server]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [client]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [examples]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [webapp]. [INFO] There are no new bugs found totally]. [TRACE] SpotBugs diffs checked and reports created [TRACE] Summary file size is 2535 bytes [TRACE] Full summary file size is 1525 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Running test-patch task MODERNIZER Running modernizer op=report [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-clean.txt [TRACE] Modernizer errors before patch: 52 [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-patch.txt [TRACE] Modernizer errors after patch: 52 Testing JIRA OOZIE-3586 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 +1 the patch adds/modifies 2 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings +1 There are no new bugs found in total. +1 There are no new bugs found in [fluent-job/fluent-job-api]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [core]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [webapp]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 1445 +1 DISTRO +1 distro tarball builds with the patch +1 MODERNIZER +1 Overall result, good!, no -1s The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1314/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 32090 0 100 3209 0 2410 0:00:01 0:00:01 --:--:--
[jira] [Commented] (OOZIE-3586) Oozie spark actions using --keytab fail due to duplicate dist. cache
[ https://issues.apache.org/jira/browse/OOZIE-3586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17054891#comment-17054891 ] Hadoop QA commented on OOZIE-3586: -- PreCommit-OOZIE-Build started > Oozie spark actions using --keytab fail due to duplicate dist. cache > > > Key: OOZIE-3586 > URL: https://issues.apache.org/jira/browse/OOZIE-3586 > Project: Oozie > Issue Type: Bug >Reporter: Janos Makai >Assignee: Janos Makai >Priority: Major > Attachments: OOZIE-3586-001.patch, OOZIE-3586-002.patch, > OOZIE-3586-003.patch > > > In CDH 6.x, on the Hadoop 3 rebase, it is now a failure if items are added > multiple times to the distributed cache. In CDH5, this was a warning. This is > not an issue for most uses, as adding multiple times typically is user error, > but this completely breaks Spark actions with keytabs (--keytab). > Oozie spark actions add everything in the distributed cache of the launcher > job to the distributed cache of the spark job, meaning the keytab is already > there, then the --keytab argument tries to add it again causing the failure > [0]. > [0] > Caused by: java.lang.IllegalArgumentException: Attempt to add > (.../filename.keytab#filename.keytab) multiple times to the distributed cache. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (93 issues) Subscriber: ooziedaily Key Summary OOZIE-3592 Do not print misleading System.exit() exception for successful jobs https://issues.apache.org/jira/browse/OOZIE-3592 OOZIE-3586 Oozie spark actions using --keytab fail due to duplicate dist. cache https://issues.apache.org/jira/browse/OOZIE-3586 OOZIE-3568 Have large amount of log information “WARN messages [main] openjpa.MetaData” in jetty.log need to clean https://issues.apache.org/jira/browse/OOZIE-3568 OOZIE-3567 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3567 OOZIE-3560 IDEA shows have some error in index.jsp https://issues.apache.org/jira/browse/OOZIE-3560 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3434 Filtering for invalid jobtype should give error message https://issues.apache.org/jira/browse/OOZIE-3434 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3254 [coordinator] LAST_ONLY and NONE execution modes: possible OutOfMemoryError when there are too many coordinator actions to materialize https://issues.apache.org/jira/browse/OOZIE-3254 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging
