[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (93 issues) Subscriber: ooziedaily Key Summary OOZIE-3522 Migrate from Guava's Joiner https://issues.apache.org/jira/browse/OOZIE-3522 OOZIE-3506 Flaky test TestOozieRollingPolicy https://issues.apache.org/jira/browse/OOZIE-3506 OOZIE-3486 duplicate code in ControlNodeHandler https://issues.apache.org/jira/browse/OOZIE-3486 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3476 Migrate several classes to JUnit4 https://issues.apache.org/jira/browse/OOZIE-3476 OOZIE-3468 Use modernizer plugin https://issues.apache.org/jira/browse/OOZIE-3468 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3265 properties RERUN_FAIL_NODES and RERUN_SKIP_NODES should be able to appear together https://issues.apache.org/jira/browse/OOZIE-3265 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3179 Adding a configurable config-default.xml location to a workflow https://issues.apache.org/jira/browse/OOZIE-3179 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib dire
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[
https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876160#comment-16876160
]
Hadoop QA commented on OOZIE-3522:
--
Testing JIRA OOZIE-3522
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [19] new bugs found below threshold in total
that must be fixed.
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:red}-1{color} There are [15] new bugs found below threshold in
[tools] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
.At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
.At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
.At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
.At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:red}-1{color} There are [4] new bugs found below threshold in
[core] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The most important SpotBugs errors are:
.At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
.At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
.This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
.At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3165
.{color:orange}Tests failed at first run:{color}
TestCoordChangeXCommand#testCoordChangeEndTime3
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1159/
> Migrate fr
Failed: OOZIE-3522 PreCommit Build #1159
Jira: https://issues.apache.org/jira/browse/OOZIE-3522
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1159/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 2.03 MB...]
[TRACE] Summary file size is 3965 bytes
[TRACE] Full summary file size is 8189 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Testing JIRA OOZIE-3522
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any star imports
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 Javadoc generation succeeded with the patch
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [19] new bugs found below threshold in total that must be fixed.
+1 There are no new bugs found in [fluent-job/fluent-job-api].
-1 There are [15] new bugs found below threshold in [tools] that must be
fixed, listing only the first [5] ones.
You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
The top [5] most important SpotBugs errors are:
At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/git].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [examples].
-1 There are [4] new bugs found below threshold in [core] that must be
fixed.
You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
The most important SpotBugs errors are:
At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
+1 There are no new bugs found in [client].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [webapp].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 3165
Tests failed at first run:
TestCoordChangeXCommand#testCoordChangeEndTime3
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/1159/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0100 48100 0 100 4810 0 4217 0:00:01 0:00:01 --:--:--
4219{"self":"https://issues.apache.org/jira/rest/api/2/iss
[jira] [Commented] (OOZIE-3468) Use modernizer plugin
[
https://issues.apache.org/jira/browse/OOZIE-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876151#comment-16876151
]
Andras Salamon commented on OOZIE-3468:
---
Looking better:
{noformat}
$ mvn modernizer:modernizer | grep -i ERROR | sed -E 's/.*(Prefer .*)/\1/g' |
sort | uniq -c | sort -nr
56 Prefer java.lang.StringBuilder
34 Prefer java.util.Collections.emptyList()
26 Prefer java.util.Collections.emptyMap()
26 Prefer java.lang.Long.valueOf(long)
21 Prefer java.lang.Integer.valueOf(int)
16 Prefer java.io.InputStreamReader(java.io.InputStream,
java.nio.charset.Charset)
11 Prefer java.lang.String.(byte[], java.nio.charset.Charset)
6 Prefer java.lang.String.getBytes(java.nio.charset.Charset)
5 Prefer javax.inject.Provider
4 Prefer java.io.OutputStreamWriter(java.io.OutputStream,
java.nio.charset.Charset)
3 Prefer java.util.LinkedHashSet<>()
2 Prefer java.util.LinkedHashMap<>()
1 Prefer java.util.HashMap<>()
1 Prefer java.util.Base64.Encoder.encodeToString(byte[])
1 Prefer java.util.Base64.Decoder.decode(String)
1 Prefer java.lang.Double.valueOf(double){noformat}
> Use modernizer plugin
> -
>
> Key: OOZIE-3468
> URL: https://issues.apache.org/jira/browse/OOZIE-3468
> Project: Oozie
> Issue Type: Improvement
> Components: build
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3468-01-wip.patch
>
>
> Recently I've opened a few jiras which suggested to use standard java classes
> instead of external libraries ( OOZIE-3463, OOZIE-3467). There is a tool
> which can find such technical depts: [maven modernizer
> plugin|https://github.com/gaul/modernizer-maven-plugin].
> The usage is quite simple:
> {noformat}
> $ mvn modernizer:modernizer
> ...
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:88:
> Prefer java.lang.String.(byte[], java.nio.charset.Charset)
>
> [ERROR] /src/oozie/core/src/main/java/org/apache/oozie/StringBlob.java:122:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:85:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V2ValidateServlet.java:92:
> Prefer java.nio.charset.StandardCharsets
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1JobsServlet.java:188:
> Prefer java.util.ArrayList<>()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:91:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:101:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:110:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/JVMInfo.java:119:
> Prefer java.lang.StringBuilder
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/VersionServlet.java:36:
> Prefer java.util.Collections.emptyList()
> [ERROR]
> /src/oozie/core/src/main/java/org/apache/oozie/servlet/V1AdminServlet.java:49:
> Prefer java.util.Collections.emptyList()
> ...
> {noformat}
> There are several ways to use this plugin:
> # Add the plugin to the root pom and let developers manually use this
> plugin. It's the simplest solution, but it will be easy to forget it.
> # Add this to the precommit script similarly to findbugs and at least avoid
> to insert new code which uses old style API. Probably we will have the same
> problems like we have with findbugs, we will have lots of false positive
> warnings.
> # Turn the plugin on by default and fail the compilation if it finds any
> problem. I think this is too strict.
> If we choose option 2 or 3 we should probably specify an ignore list, I don't
> think for instance that we really want to change all the {{new Long(10)}}
> code to {{Long.valueOf(10)}}.
> By default this plugin checks the target java version (1.8 right now) but
> it's possible to specify 1.7 instead if we want to focus on those problems
> first. (It was not working for me without specifying the java version.)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3445) [build] Replace findbugs-diff jar
[ https://issues.apache.org/jira/browse/OOZIE-3445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876140#comment-16876140 ] Andras Salamon commented on OOZIE-3445: --- We should use the text output of spotbugs, as described here: [https://github.com/spotbugs/discuss/issues/55] > [build] Replace findbugs-diff jar > - > > Key: OOZIE-3445 > URL: https://issues.apache.org/jira/browse/OOZIE-3445 > Project: Oozie > Issue Type: Task >Reporter: Julia Kinga Marton >Assignee: Julia Kinga Marton >Priority: Minor > > During precommit, we use an extrenal[ > jar|https://repo1.maven.org/maven2/me/andrz/findbugs/findbugs-diff/0.1.0/]. > We should search for alternatives and replace it, because it not so nice to > store the md5, hardwiring the jar address and downloading the jar again and > again. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876103#comment-16876103 ] Andras Salamon commented on OOZIE-3522: --- Something went wrong during the precommit, retriggering. > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876102#comment-16876102 ] Hadoop QA commented on OOZIE-3522: -- PreCommit-OOZIE-Build started > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Failed: OOZIE-3522 PreCommit Build #1158
Jira: https://issues.apache.org/jira/browse/OOZIE-3522
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1158/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 2.00 MB...]
xargs: WARNING: a NUL character occurred in the input. It cannot be passed
through in the argument list. Did you mean to use the --null option?
Running test-patch task DISTRO
Testing JIRA OOZIE-3522
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any star imports
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 Javadoc generation succeeded with the patch
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [22] new bugs found below threshold in total that must be fixed.
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/git].
+1 There are no new bugs found in [client].
+1 There are no new bugs found in [docs].
-1 There are [15] new bugs found below threshold in [tools] that must be
fixed, listing only the first [5] ones.
You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
The top [5] most important SpotBugs errors are:
At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
+1 There are no new bugs found in [fluent-job/fluent-job-api].
0 There are [4] new bugs found in [server] that would be nice to have fixed.
You can find the SpotBugs diff here: server/findbugs-new.html
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [examples].
-1 There are [7] new bugs found below threshold in [core] that must be
fixed, listing only the first [5] ones.
You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
The top [5] most important SpotBugs errors are:
At BulkJPAExecutor.java:[line 207]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
At BulkJPAExecutor.java:[line 177]: At BulkJPAExecutor.java:[line 176]
At BulkJPAExecutor.java:[line 206]: At BulkJPAExecutor.java:[line 200]
This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 207]
At BulkJPAExecutor.java:[line 112]: At BulkJPAExecutor.java:[line 128]
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
-1 TESTS
Tests run: 2967
Tests failed : 1
Tests in error : 0
Tests timed out : 0
-1 [ERROR] There are [1] test failures in [core]. Listing only the first [5]
ones
testPurgeableWorkflowPurgeableSubWorkflowPurgeableSubSubWorkflow:org.apache.oozie.command.TestPurgeXCommand
Check console output for the full list of errors/failures
+1 DISTRO
+1 distro tarball builds with the patch
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/1158/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0100 51370 0 100 5137 0 4411 0:00:01 0:00:01 --:--:--
4413{"self":"https://issues.apache.org/jira/rest/api
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[
https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876076#comment-16876076
]
Hadoop QA commented on OOZIE-3522:
--
Testing JIRA OOZIE-3522
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [22] new bugs found below threshold in total
that must be fixed.
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:red}-1{color} There are [15] new bugs found below threshold in
[tools] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
.At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
.At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
.At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
.At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:orange}0{color} There are [4] new bugs found in [server] that would
be nice to have fixed.
.You can find the SpotBugs diff here: server/findbugs-new.html
.{color:green}+1{color} There are no new bugs found in [webapp].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:red}-1{color} There are [7] new bugs found below threshold in
[core] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At BulkJPAExecutor.java:[line 207]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 177]: At BulkJPAExecutor.java:[line 176]
.At BulkJPAExecutor.java:[line 206]: At BulkJPAExecutor.java:[line 200]
.This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 207]
.At BulkJPAExecutor.java:[line 112]: At BulkJPAExecutor.java:[line 128]
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 2967
.Tests failed : 1
.Tests in error : 0
.Tests timed out : 0
{color:red}-1{color} [ERROR] There are [1] test failures in [core]. Listing
only the first [5] ones
testPurgeableWorkflowPurgeableSubWorkflowPurgeableSubSubWorkflow:org.apache.oozie.command.TestPurgeXCommand
Check console output for the full list of errors/failures
{color
[jira] [Commented] (OOZIE-3520) Upgrade Derby to 10.14.2.0
[ https://issues.apache.org/jira/browse/OOZIE-3520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876050#comment-16876050 ] Andras Salamon commented on OOZIE-3520: --- Thanks for the contribution [~nobigo], +1, committed to master. > Upgrade Derby to 10.14.2.0 > -- > > Key: OOZIE-3520 > URL: https://issues.apache.org/jira/browse/OOZIE-3520 > Project: Oozie > Issue Type: Bug >Affects Versions: trunk >Reporter: Andras Salamon >Assignee: duan xiong >Priority: Major > Attachments: OOZIE-3520-001.patch > > > Upgrade Derby to the latest in the 10.14.x line: 10.14.2.0. > We cannot upgrade to 10.15, since it requires JDK9: > [https://db.apache.org/derby/derby_downloads.html] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[
https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876045#comment-16876045
]
Julia Kinga Marton commented on OOZIE-3513:
---
Thank you @asalamon for the contribution! +1, committed to master
> Migrate from Preconditions.checkNotNull and ParamChecker.notNull
>
>
> Key: OOZIE-3513
> URL: https://issues.apache.org/jira/browse/OOZIE-3513
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch,
> OOZIE-3513-03.patch, OOZIE-3513-04.patch, OOZIE-3513-05.patch
>
>
> We currently use both Guava's {{Preconditions.checkNotNull}} and our own
> {{ParamChecker.notNull}} to check for null arguments. Instead we should use
> the standard {{Objects.requireNonNull}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Re: Review Request 70964: OOZIE-3513 Migrate from Preconditions.checkNotNull and ParamChecker.notNull
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/70964/#review216267 --- Ship it! Ship It! - Kinga Marton On June 28, 2019, 8:57 a.m., Andras Salamon wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/70964/ > --- > > (Updated June 28, 2019, 8:57 a.m.) > > > Review request for oozie and Kinga Marton. > > > Repository: oozie-git > > > Description > --- > > OOZIE-3513 Migrate from Preconditions.checkNotNull and ParamChecker.notNull > > > Diffs > - > > client/src/main/java/org/apache/oozie/client/ApiJarFactory.java bb85d6780 > client/src/main/java/org/apache/oozie/client/ApiJarLoader.java 6ecf485ed > core/src/main/java/org/apache/oozie/BaseLocalOozieClient.java 376ea1165 > core/src/main/java/org/apache/oozie/XException.java fb1a381bb > core/src/main/java/org/apache/oozie/action/ActionExecutorException.java > 9bc0e81b9 > core/src/main/java/org/apache/oozie/action/hadoop/GitActionExecutor.java > fc4d65b83 > core/src/main/java/org/apache/oozie/action/hadoop/HadoopTokenHelper.java > 1bdeb32f6 > core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java > ec45fe495 > > core/src/main/java/org/apache/oozie/action/hadoop/MapReduceActionExecutor.java > a509e4d98 > core/src/main/java/org/apache/oozie/action/hadoop/ShareLibExcluder.java > 9c45e5392 > core/src/main/java/org/apache/oozie/command/TransitionXCommand.java > 262a78f5b > > core/src/main/java/org/apache/oozie/command/bundle/BundleJobResumeXCommand.java > bc58be6ef > > core/src/main/java/org/apache/oozie/command/bundle/BundleSubmitXCommand.java > ab88cac6c > > core/src/main/java/org/apache/oozie/command/coord/CoordActionNotificationXCommand.java > d51f0d780 > > core/src/main/java/org/apache/oozie/command/coord/CoordActionSkipXCommand.java > ef0fc2d3e > > core/src/main/java/org/apache/oozie/command/coord/CoordActionTimeOutXCommand.java > 9646d738a > > core/src/main/java/org/apache/oozie/command/coord/CoordActionsIgnoreXCommand.java > 88842f1f1 > core/src/main/java/org/apache/oozie/command/coord/CoordRerunXCommand.java > 2f158cdbf > core/src/main/java/org/apache/oozie/command/coord/CoordSubmitXCommand.java > 2622f05d9 > core/src/main/java/org/apache/oozie/command/wf/ReRunXCommand.java dd8d3d2f6 > core/src/main/java/org/apache/oozie/command/wf/SubmitHttpXCommand.java > 9ee84063a > core/src/main/java/org/apache/oozie/command/wf/SubmitXCommand.java > 70b9adc1c > > core/src/main/java/org/apache/oozie/command/wf/WorkflowNotificationXCommand.java > 62bf9b5b3 > core/src/main/java/org/apache/oozie/coord/CoordELFunctions.java 2761e44b0 > core/src/main/java/org/apache/oozie/coord/CoordUtils.java 1d97accd4 > core/src/main/java/org/apache/oozie/executor/jpa/BulkJPAExecutor.java > 42961157a > > core/src/main/java/org/apache/oozie/executor/jpa/BundleActionGetJPAExecutor.java > 3293efd69 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleActionInsertJPAExecutor.java > 4b99c1a59 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleActionsCountForJobGetJPAExecutor.java > c9f3a212d > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobGetCoordinatorsJPAExecutor.java > 2f4665e15 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobGetForUserJPAExecutor.java > aef9eb79c > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobGetJPAExecutor.java > 7f98cd2a8 > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobInfoGetJPAExecutor.java > 6e1ea3e5b > > core/src/main/java/org/apache/oozie/executor/jpa/BundleJobInsertJPAExecutor.java > e7373d67d > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForCheckJPAExecutor.java > 9e8b33191 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForExternalIdJPAExecutor.java > 61614e9fa > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInfoJPAExecutor.java > 211d594e5 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForInputCheckJPAExecutor.java > 484ff93c9 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForStartJPAExecutor.java > 14b831d65 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetForTimeoutJPAExecutor.java > c720be99d > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionGetJPAExecutor.java > 53214ea64 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionInsertJPAExecutor.java > 09e885793 > > core/src/main/java/org/apache/oozie/executor/jpa/CoordActionRemoveJPAExecutor.java > 8ea31a31d > > core/src/main/java/org/apache/ooz
[jira] [Commented] (OOZIE-3522) Migrate from Guava's Joiner
[ https://issues.apache.org/jira/browse/OOZIE-3522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876023#comment-16876023 ] Hadoop QA commented on OOZIE-3522: -- PreCommit-OOZIE-Build started > Migrate from Guava's Joiner > --- > > Key: OOZIE-3522 > URL: https://issues.apache.org/jira/browse/OOZIE-3522 > Project: Oozie > Issue Type: Sub-task >Reporter: Andras Salamon >Assignee: Andras Salamon >Priority: Major > Attachments: OOZIE-3522-01.patch > > > As mentioned in OOZIE-3488 we might use standard JDK classes instead of the > [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] > class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (OOZIE-3522) Migrate from Guava's Joiner
Andras Salamon created OOZIE-3522: - Summary: Migrate from Guava's Joiner Key: OOZIE-3522 URL: https://issues.apache.org/jira/browse/OOZIE-3522 Project: Oozie Issue Type: Sub-task Reporter: Andras Salamon Assignee: Andras Salamon As mentioned in OOZIE-3488 we might use standard JDK classes instead of the [Joiner|https://guava.dev/releases/11.0.2/api/docs/index.html?com/google/common/base/Joiner.MapJoiner.html] class of Guava as suggested here: https://stackoverflow.com/a/22577565/21348 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3488) Migrate from guava classes to the base Java implementations
[ https://issues.apache.org/jira/browse/OOZIE-3488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon updated OOZIE-3488: -- Issue Type: Sub-task (was: Improvement) Parent: OOZIE-3521 > Migrate from guava classes to the base Java implementations > --- > > Key: OOZIE-3488 > URL: https://issues.apache.org/jira/browse/OOZIE-3488 > Project: Oozie > Issue Type: Sub-task >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Labels: code-cleanup > Fix For: 5.2.0 > > Attachments: OOZIE-3488-2.patch, OOZIE-3488-3.patch, OOZIE-3488.patch > > > In Oozie, guava classes are used even when Java 8 already provides a > compatible implementation, removing these usages would reduce the dependency > to guava (which is regularly a headache due to her backward incompatible > changes) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (OOZIE-3513) Migrate from Preconditions.checkNotNull and ParamChecker.notNull
[
https://issues.apache.org/jira/browse/OOZIE-3513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated OOZIE-3513:
--
Issue Type: Sub-task (was: Bug)
Parent: OOZIE-3521
> Migrate from Preconditions.checkNotNull and ParamChecker.notNull
>
>
> Key: OOZIE-3513
> URL: https://issues.apache.org/jira/browse/OOZIE-3513
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3513-01.patch, OOZIE-3513-02.patch,
> OOZIE-3513-03.patch, OOZIE-3513-04.patch, OOZIE-3513-05.patch
>
>
> We currently use both Guava's {{Preconditions.checkNotNull}} and our own
> {{ParamChecker.notNull}} to check for null arguments. Instead we should use
> the standard {{Objects.requireNonNull}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3467) Migrate from com.google.common.base.Stopwatch
[
https://issues.apache.org/jira/browse/OOZIE-3467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated OOZIE-3467:
--
Issue Type: Sub-task (was: Improvement)
Parent: OOZIE-3521
> Migrate from com.google.common.base.Stopwatch
> -
>
> Key: OOZIE-3467
> URL: https://issues.apache.org/jira/browse/OOZIE-3467
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3467-01.patch
>
>
> OOZIE-1826 created {{JvmPauseMonitorService}} which uses Guava's stopwatch.
> The implementation was heavily inspired by HADOOP-9618. Guava changed the
> StopWatch class in a way that we either create a version which is compatible
> in the old (Guava 11) or the new (Guava 27) interface. To make the code Guava
> version independent we should not use this class anymore. Hadoop also
> replaced this class in HADOOP-11032.
> We don't really need nanosecond accuracy here, we only read milliseconds from
> the stopwatch and the default thresholds are 1 second
> (oozie.service.JvmPauseMonitorService.info-threshold.ms = 1000ms) and 10
> seconds (oozie.service.JvmPauseMonitorService.warn-threshold.ms = 10 000ms)
> so even millisecond is a bit overkill. We could just use
> {{System.currentTimeMillis()}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (OOZIE-3466) Migrate from com.google.common.io.Closeables to org.apache.commons.io.IOUtils
[
https://issues.apache.org/jira/browse/OOZIE-3466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated OOZIE-3466:
--
Issue Type: Sub-task (was: Improvement)
Parent: OOZIE-3521
> Migrate from com.google.common.io.Closeables to org.apache.commons.io.IOUtils
> -
>
> Key: OOZIE-3466
> URL: https://issues.apache.org/jira/browse/OOZIE-3466
> Project: Oozie
> Issue Type: Sub-task
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Andras Salamon
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3466-01.patch
>
>
> [JavaActionExecutor|https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java]
> uses both Guava's Closeables and commons-io's IOUtils to close the
> {{yarnClient}} quietly. The Guava's versions causes problem in the Guava
> update (OOZIE-3418). We should use only commons-io's IOUtils in the code.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (OOZIE-3521) [Umbrella] Reduce Guava usage
Andras Salamon created OOZIE-3521: - Summary: [Umbrella] Reduce Guava usage Key: OOZIE-3521 URL: https://issues.apache.org/jira/browse/OOZIE-3521 Project: Oozie Issue Type: Bug Reporter: Andras Salamon In the last weeks we have resolved a few jiras which reduced the guava usage. Let's create an umbrella Jira to collect these jiras and discuss further ideas. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
