Failed: OOZIE-3535 PreCommit Build #1204
Jira: https://issues.apache.org/jira/browse/OOZIE-3535 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1204/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.03 MB...] [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [DEBUG] There are no new bugs found in [examples]. [TRACE] New XMLLib present, calling 'xmllint --xpath' to get bug instance counts [ERROR] There are [7] new bugs found below threshold in [core]. [DEBUG] You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html [ERROR] There are [7] new bugs found below threshold in total that must be fixed. [TRACE] SpotBugs diffs checked and reports created [TRACE] Summary file size is 3381 bytes [TRACE] Full summary file size is 4275 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Testing JIRA OOZIE-3535 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN +1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 +1 the patch adds/modifies 1 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [7] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [docs]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [fluent-job/fluent-job-api]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [webapp]. +1 There are no new bugs found in [examples]. -1 There are [7] new bugs found below threshold in [core] that must be fixed, listing only the first [5] ones. You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html The top [5] most important SpotBugs errors are: At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175] At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199] This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206] At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127] +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 3175 Tests failed at first run: TestPurgeXCommand#testPurgeableWorkflowPurgeableSubWorkflowPurgeableSubSubWorkflow For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1204/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902065#comment-16902065
]
Julia Kinga Marton commented on OOZIE-3468:
---
[~asalamon74], I have some small comments related the actual patch:
* in the modernizer script the PATCHFILE option is not used. can you please
remove it?
* _"REPORT+=("\{color:red}-1\{color} the patch introduces ${newErrorInClass}
new warnings in ${class_name}")",_ please replace warning with warning(s).
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Commented] (OOZIE-3535) mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full effect
[
https://issues.apache.org/jira/browse/OOZIE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902015#comment-16902015
]
Hadoop QA commented on OOZIE-3535:
--
PreCommit-OOZIE-Build started
> mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full
> effect
>
>
> Key: OOZIE-3535
> URL: https://issues.apache.org/jira/browse/OOZIE-3535
> Project: Oozie
> Issue Type: Bug
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3535-001.patch
>
>
> mapreduce.job.acl-view-job property in Oozie workflow.xml will be ignored if
> only {{yarn.acl.enable}} = true, but {{mapreduce.cluster.acls.enabled}} =
> false.
> This is caused by the following check:
> [https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/YarnACLHandler.java#L39]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Comment Edited] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16891862#comment-16891862
]
Julia Kinga Marton edited comment on OOZIE-3468 at 8/7/19 11:59 AM:
[~asalamon74] can you please check your code with ShellCheck and fix the
reported issues?
was (Author: kmarton):
[~asalamon74] can you please check tour code with ShellCheck and fix the
reported issues?
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch, OOZIE-3468-02-wip.patch,
> OOZIE-3468-03-wip.patch, OOZIE-3468-04.patch, OOZIE-3468-06.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (OOZIE-3535) mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full effect
[
https://issues.apache.org/jira/browse/OOZIE-3535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Julia Kinga Marton updated OOZIE-3535:
--
Attachment: OOZIE-3535-001.patch
> mapreduce.job.acl-view-job property in Oozie workflow.xml not taking full
> effect
>
>
> Key: OOZIE-3535
> URL: https://issues.apache.org/jira/browse/OOZIE-3535
> Project: Oozie
> Issue Type: Bug
>Reporter: Julia Kinga Marton
>Assignee: Julia Kinga Marton
>Priority: Major
> Attachments: OOZIE-3535-001.patch
>
>
> mapreduce.job.acl-view-job property in Oozie workflow.xml will be ignored if
> only {{yarn.acl.enable}} = true, but {{mapreduce.cluster.acls.enabled}} =
> false.
> This is caused by the following check:
> [https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/YarnACLHandler.java#L39]
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (91 issues) Subscriber: ooziedaily Key Summary OOZIE-3529 Oozie not supported for s3 as filesystem https://issues.apache.org/jira/browse/OOZIE-3529 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3468 Use modernizer plugin https://issues.apache.org/jira/browse/OOZIE-3468 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3254 [coordinator] LAST_ONLY and NONE execution modes: possible OutOfMemoryError when there are too many coordinator actions to materialize https://issues.apache.org/jira/browse/OOZIE-3254 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression Language Evaulator
