[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (91 issues) Subscriber: ooziedaily Key Summary OOZIE-3542 Handle better old Hdfs implementations in ECPolicyDisabler https://issues.apache.org/jira/browse/OOZIE-3542 OOZIE-3536 oozie-main(pom.xml) plugin maven-javadoc-plugin upgrade version caused configuration can't find the Tag https://issues.apache.org/jira/browse/OOZIE-3536 OOZIE-3529 Oozie not supported for s3 as filesystem https://issues.apache.org/jira/browse/OOZIE-3529 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3254 [coordinator] LAST_ONLY and NONE execution modes: possible OutOfMemoryError when there are too many coordinator actions to materialize https://issues.apache.org/jira/browse/OOZIE-3254 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging https://issues.apache.org/jira/browse/OOZIE-2795 OOZIE-2784 Include WEEK as a parameter in the Coordinator Expression
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936957#comment-16936957 ] Zsombor Gegesy commented on OOZIE-3542: --- Yes, the problem was, that the there are multiple reflective calls to the remote HDFS service, where the same error could happen - and unfortunately only one call is handled previously. The new patch adds the same checks to all calls. > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936909#comment-16936909 ] Julia Kinga Marton commented on OOZIE-3542: --- [~zsombor], can you please share what was the problem with the initial patch? > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[
https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936888#comment-16936888
]
Hadoop QA commented on OOZIE-3542:
--
Testing JIRA OOZIE-3542
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:green}+1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [15] new bugs found below threshold in total
that must be fixed.
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [webapp].
.{color:green}+1{color} There are no new bugs found in [core].
.{color:red}-1{color} There are [15] new bugs found below threshold in
[tools] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
.At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
.At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
.At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
.At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
.{color:green}+1{color} There are no new bugs found in [client].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3182
.{color:orange}Tests failed at first run:{color}
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1223/
> Handle better old Hdfs implementations in ECPolicyDisabler
> --
>
> Key: OOZIE-3542
> URL: https://issues.apache.org/jira/browse/OOZIE-3542
> Project: Oozie
> Issue Type: Bug
> Components: tools
>Reporter: Zsombor Gegesy
>Assignee: Zsombor Gegesy
>Priority: Major
> Fix For: 5.2.0
>
> Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch,
> OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch
>
>
> Currently, ECPolicyDisabler checks if the local hdfs implementation has the
> necessary methods to get and set erasure coding policy. However, if the
> namenode implemen
Failed: OOZIE-3542 PreCommit Build #1223
Jira: https://issues.apache.org/jira/browse/OOZIE-3542
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1223/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 3.11 MB...]
[TRACE] Summary file size is 3193 bytes
[TRACE] Full summary file size is 6600 bytes
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar]
removed
[TRACE] File
[/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5]
removed
Running test-patch task BACKWARDS_COMPATIBILITY
Running test-patch task TESTS
Running test-patch task DISTRO
Running test-patch task MODERNIZER
Running modernizer op=report
[TRACE] grep -c '.*ERROR.*Prefer'
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-clean.txt
[TRACE] Modernizer errors before patch: 69
[TRACE] grep -c '.*ERROR.*Prefer'
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-patch.txt
[TRACE] Modernizer errors after patch: 69
Testing JIRA OOZIE-3542
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
+1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any star imports
+1 the patch does not introduce any line longer than 132
+1 the patch adds/modifies 1 testcase(s)
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 Javadoc generation succeeded with the patch
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [15] new bugs found below threshold in total that must be fixed.
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [fluent-job/fluent-job-api].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/git].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [server].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [core].
-1 There are [15] new bugs found below threshold in [tools] that must be
fixed, listing only the first [5] ones.
You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
The top [5] most important SpotBugs errors are:
At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
+1 There are no new bugs found in [client].
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 3182
Tests failed at first run:
TestCoordActionsKillXCommand#testActionKillCommandActionNumbers
For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
+1 DISTRO
+1 distro tarball builds with the patch
+1 MODERNIZER
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/1223/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0{"self":"https://issues.apache.org/jira/rest/api/2/issue/13255219/comment/16936888","id":"16936888","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://issues.apache.org/jira/secure/useravatar?ownerId=hadoopqa&avatarId=10393","24x24":"https://issues.apache.org/jira/secure/userava
[jira] [Assigned] (OOZIE-3465) Migrate from commons-codec
[
https://issues.apache.org/jira/browse/OOZIE-3465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz reassigned OOZIE-3465:
--
Assignee: Mate Juhasz
> Migrate from commons-codec
> --
>
> Key: OOZIE-3465
> URL: https://issues.apache.org/jira/browse/OOZIE-3465
> Project: Oozie
> Issue Type: Improvement
>Affects Versions: trunk
>Reporter: Andras Salamon
>Assignee: Mate Juhasz
>Priority: Major
>
> Ooize uses {{commons-codec}} 1.4 which is almost [10
> years|https://commons.apache.org/proper/commons-codec/changes-report.html]
> old. We hardly use this library I've only found a few references to the
> {{Base64}} class in Oozie. Java8 defines a
> [Base64|https://docs.oracle.com/javase/8/docs/api/java/util/Base64.html]
> class which we could use instead.
> We should be careful to put this jar into the sharelibs if and only if some
> other library requires it.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-2764) Action failures related to HADOOP_CLASSPATH when using Hadoop 2.7.3
[
https://issues.apache.org/jira/browse/OOZIE-2764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated OOZIE-2764:
--
Priority: Critical (was: Blocker)
> Action failures related to HADOOP_CLASSPATH when using Hadoop 2.7.3
> ---
>
> Key: OOZIE-2764
> URL: https://issues.apache.org/jira/browse/OOZIE-2764
> Project: Oozie
> Issue Type: Bug
> Components: action, core
>Affects Versions: 4.3.0
> Environment: Hadoop 2.7.3, Oozie 4.3.0, Hive 2.1.0, Pig 0.16.0, Spark
> 1.6.1
>Reporter: Alexandre Linte
>Priority: Critical
>
> Every action tested (MapReduce, Java, Spark, Hive, Pig) are failing with a
> "java.lang.NoSuchFieldError: HADOOP_CLASSPATH".
> Oozie has been compiled with the command:
> {noformat}
> $ bin/mkdistro.sh -DskipTests -Phadoop-2 -Dhadoop.version=2.7.3
> -Dpig.version=0.16.0 -Dspark.version=1.6.1
> {noformat}
> The error stacktrace is below:
> {noformat}
> 2016-12-26T14:11:16+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.service.CoordMaterializeTriggerService$CoordMaterializeTriggerRunnableUSER[-]
> GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] CoordMaterializeTriggerService -
> Curr Date= 2016-12-26T13:16Z, Num jobs to materialize = 0
> 2016-12-26T14:11:16+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.service.CoordMaterializeTriggerService$CoordMaterializeTriggerRunnableUSER[-]
> GROUP[-] TOKEN[-] APP[-] JOB[-] ACTION[-] Released lock for
> [org.apache.oozie.service.CoordMaterializeTriggerService]
> 2016-12-26T14:11:16+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.service.StatusTransitService$StatusTransitRunnableReleased
> lock for [org.apache.oozie.service.StatusTransitService]
> 2016-12-26T14:11:16+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.command.PurgeXCommandUSER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] STARTED Purge to purge Workflow Jobs older than [30] days,
> Coordinator Jobs older than [7] days, and Bundlejobs older than [7] days.
> 2016-12-26T14:11:16+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.command.PurgeXCommandUSER[-] GROUP[-] TOKEN[-] APP[-] JOB[-]
> ACTION[-] ENDED Purge deleted [0] workflows, [0] coordinatorActions, [0]
> coordinators, [0] bundles
> 2016-12-26T14:11:16+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.service.PauseTransitServiceReleased lock for
> [org.apache.oozie.service.PauseTransitService]
> 2016-12-26T14:11:17+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.command.wf.ActionStartXCommandUSER[shfs3453] GROUP[-]
> TOKEN[] APP[SparkPi-test] JOB[000-161226114905740-oozie-W]
> ACTION[000-161226114905740-oozie-W@spark-node] Start action
> [000-161226114905740-oozie-W@spark-node] with user-retry state :
> userRetryCount [0], userRetryMax [3], userRetryInterval [1]
> 2016-12-26T14:11:17+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.service.HadoopAccessorServiceUSER[shfs3453] GROUP[-] TOKEN[]
> APP[SparkPi-test] JOB[000-161226114905740-oozie-W]
> ACTION[000-161226114905740-oozie-W@spark-node] Processing configuration
> file [/opt/application/Oozie/current/conf/action-conf/default.xml] for action
> [default] and hostPort [*]
> 2016-12-26T14:11:17+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.service.HadoopAccessorServiceUSER[shfs3453] GROUP[-] TOKEN[]
> APP[SparkPi-test] JOB[000-161226114905740-oozie-W]
> ACTION[000-161226114905740-oozie-W@spark-node] Processing configuration
> file [/opt/application/Oozie/current/conf/action-conf/spark.xml] for action
> [spark] and hostPort [*]
> 2016-12-26T14:11:22+01:00 oozie01.bigdata.fr oozie INFO -
> org.apache.oozie.service.HadoopAccessorServiceUSER[shfs3453] GROUP[-] TOKEN[]
> APP[SparkPi-test] JOB[000-161226114905740-oozie-W]
> ACTION[000-161226114905740-oozie-W@spark-node] Delegation Token Renewer
> details:
> Principal=rm/_HOST@SANDBOX.HADOOP,Target=sandbox-RMS:8032,Renewer=rm/sandbox-rms@SANDBOX.HADOOP
> 2016-12-26T14:11:22+01:00 oozie01.bigdata.fr oozie WARN -
> org.apache.hadoop.mapreduce.JobResourceUploaderHadoop command-line option
> parsing not performed. Implement the Tool interface and execute your
> application with ToolRunner to remedy this.
> 2016-12-26T14:11:22+01:00 oozie01.bigdata.fr oozie WARN -
> org.apache.hadoop.mapreduce.JobResourceUploaderNo job jar file set. User
> classes may not be found. See Job or Job#setJar(String).
> 2016-12-26T14:11:23+01:00 oozie01.bigdata.fr oozie ERROR -
> org.apache.oozie.command.wf.ActionStartXCommandUSER[shfs3453] GROUP[-]
> TOKEN[] APP[SparkPi-test] JOB[000-161226114905740-oozie-W]
> ACTION[000-161226114905740-oozie-W@spark-node] Error,
> 2016-12-26T14:11:23.573405+01:00 localhost java.lang.NoSuchFieldError:
> HADOOP_CLASSPATH
> 2016-12-
[jira] [Updated] (OOZIE-1416) verify if MAPREDUCE-5199 impacts launcher jobs
[ https://issues.apache.org/jira/browse/OOZIE-1416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Salamon updated OOZIE-1416: -- Priority: Major (was: Blocker) > verify if MAPREDUCE-5199 impacts launcher jobs > -- > > Key: OOZIE-1416 > URL: https://issues.apache.org/jira/browse/OOZIE-1416 > Project: Oozie > Issue Type: Bug > Components: workflow >Affects Versions: trunk >Reporter: Alejandro Abdelnur >Priority: Major > > MAPREDUCE-5199 does aways with the tokens file. > we should verify that this does not affect how oozie works and if it does fix > things. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-2576) Oozie ssh action Cannot run program "scp"
[
https://issues.apache.org/jira/browse/OOZIE-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andras Salamon updated OOZIE-2576:
--
Priority: Major (was: Blocker)
> Oozie ssh action Cannot run program "scp"
> -
>
> Key: OOZIE-2576
> URL: https://issues.apache.org/jira/browse/OOZIE-2576
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.0.0
> Environment: Oozie server build version: 4.0.0-cdh5.2.0
>Reporter: ywheel
>Priority: Major
>
> We used oozie ssh action in a production environment, the following xml codes
> is the example:
> {noformat}
>
>
>
>
> root@192.168.1.154
> sh /tmp/test.sh
>
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
> However, we meet the errors below:
> {noformat}
> 2016-06-12 22:30:54,713 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Attempting to copy
> ssh base scripts to remote host [root@192.168.1.154]
> 2016-06-12 22:30:54,869 WARN org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Error while
> executing ssh EXECUTION
> 2016-06-12 22:30:54,870 WARN org.apache.oozie.command.wf.ActionStartXCommand:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Error starting
> action [ShellAction]. ErrorType [ERROR], ErrorCode [UNKOWN_ERROR], Message
> [UNKOWN_ERROR: Cannot run program "scp": error=2, No such file or directory]
> org.apache.oozie.action.ActionExecutorException: UNKOWN_ERROR: Cannot run
> program "scp": error=2, No such file or directory
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:599)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:204)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
> at org.apache.oozie.command.XCommand.call(XCommand.java:281)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.io.IOException: Cannot run program "scp": error=2, No such
> file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:485)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:332)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:376)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:204)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:548)
> ... 10 more
> Caused by: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.forkAndExec(Native Method)
> at java.lang.UNIXProcess.(UNIXProcess.java:135)
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
> ... 17 more
> {noformat}
> I checked the codes in {{org.apache.oozie.action.ssh.SshActionExecutor}}
> class, and I found oozie would execute 'scp' command to copy two
> files(ssh-base.sh,ssh-wrapper.sh) to the workspace folder on remote host. But
> the logs could not show any information about which file is not found.
> The workspace folder on remote host was created so the 'ssh'+ 'mkdir' command
> was executed successfully before 'scp', which means the authentication from
> oozie server to remote host is correct.
> And I tried the following command to scp that two shell files manually:
> {noformat}
> scp -o PasswordAuthentication=no -o KbdInteractiveDev
[jira] [Resolved] (OOZIE-3363) Hadoop's cleanup of local directory in uber mode causing failures for sqoop actions
[
https://issues.apache.org/jira/browse/OOZIE-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mate Juhasz resolved OOZIE-3363.
Resolution: Duplicate
> Hadoop's cleanup of local directory in uber mode causing failures for sqoop
> actions
> ---
>
> Key: OOZIE-3363
> URL: https://issues.apache.org/jira/browse/OOZIE-3363
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 4.3.0, 5.0.0
>Reporter: Clemens Valiente
>Priority: Blocker
>
> See related task OOZIE-2536 and discussion
> https://issues.apache.org/jira/browse/OOZIE-2536?focusedCommentId=16617206&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16617206
> the same issue cleaning up local directories also affects sqoop jobs that
> create the sqoop.xml file inbetween.
> {code:java}
> 2018-09-15 00:31:09,416 [AsyncDispatcher event handler] INFO
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl - Num completed Tasks: 1
> 2018-09-15 00:31:09,417 [AsyncDispatcher event handler] INFO
> org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl -
> job_1535972033593_259806Job Transitioned from RUNNING to COMMITTING
> 2018-09-15 00:31:09,419 [CommitterEvent Processor #1] INFO
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler - Processing
> the event EventType: JOB_COMMIT
> 2018-09-15 00:31:09,455 [uber-SubtaskRunner] WARN
> org.apache.hadoop.mapred.LocalContainerLauncher - Unable to delete
> unexpected local file/dir .action.xml.crc: insufficient permissions?
> 2018-09-15 00:31:09,455 [uber-SubtaskRunner] WARN
> org.apache.hadoop.mapred.LocalContainerLauncher - Unable to delete
> unexpected local file/dir .action.xml.crc: insufficient permissions?
> 2018-09-15 00:31:09,459 [CommitterEvent Processor #1] FATAL
> org.apache.hadoop.conf.Configuration - error parsing conf sqoop-site.xml
> java.io.FileNotFoundException:
> /appdata/hdfs/v7/yarn/nm/usercache/SEM/appcache/application_1535972033593_259806/container_e100_1535972033593_259806_01_01/sqoop-site.xml
> (No such file or directory)
> at java.io.FileInputStream.open(Native Method)
> at java.io.FileInputStream.(FileInputStream.java:146)
> at java.io.FileInputStream.(FileInputStream.java:101)
> at
> sun.net.www.protocol.file.FileURLConnection.connect(FileURLConnection.java:90)
> at
> sun.net.www.protocol.file.FileURLConnection.getInputStream(FileURLConnection.java:188)
> at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2483)
> at
> org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2554)
> at
> org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2507)
> at
> org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2413)
> at org.apache.hadoop.conf.Configuration.get(Configuration.java:984)
> at
> org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1034)
> at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1254)
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:804)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.touchz(CommitterEventHandler.java:268)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.handleJobCommit(CommitterEventHandler.java:282)
> at
> org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler$EventProcessor.run(CommitterEventHandler.java:237)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745){code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-2576) Oozie ssh action Cannot run program "scp"
[
https://issues.apache.org/jira/browse/OOZIE-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936818#comment-16936818
]
Mate Juhasz commented on OOZIE-2576:
Its a bit late to answer, but I think the error is because adding multiple
commands are not supported in case of ssh actions.
It should work if you specify the command like below:
{noformat}
/tmp/test.sh
{noformat}
Or:
{noformat}
sh
/tmp/test.sh
{noformat}
> Oozie ssh action Cannot run program "scp"
> -
>
> Key: OOZIE-2576
> URL: https://issues.apache.org/jira/browse/OOZIE-2576
> Project: Oozie
> Issue Type: Bug
> Components: action
>Affects Versions: 4.0.0
> Environment: Oozie server build version: 4.0.0-cdh5.2.0
>Reporter: ywheel
>Priority: Blocker
>
> We used oozie ssh action in a production environment, the following xml codes
> is the example:
> {noformat}
>
>
>
>
> root@192.168.1.154
> sh /tmp/test.sh
>
>
>
>
>
>
> Action failed, error
> message[${wf:errorMessage(wf:lastErrorNode())}]
>
>
>
> {noformat}
> However, we meet the errors below:
> {noformat}
> 2016-06-12 22:30:54,713 INFO org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Attempting to copy
> ssh base scripts to remote host [root@192.168.1.154]
> 2016-06-12 22:30:54,869 WARN org.apache.oozie.action.ssh.SshActionExecutor:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Error while
> executing ssh EXECUTION
> 2016-06-12 22:30:54,870 WARN org.apache.oozie.command.wf.ActionStartXCommand:
> SERVER[Master] USER[hdfs] GROUP[-] TOKEN[] APP[TestSsh]
> JOB[201-160113124428061-oozie-oozi-W]
> ACTION[201-160113124428061-oozie-oozi-W@ShellAction] Error starting
> action [ShellAction]. ErrorType [ERROR], ErrorCode [UNKOWN_ERROR], Message
> [UNKOWN_ERROR: Cannot run program "scp": error=2, No such file or directory]
> org.apache.oozie.action.ActionExecutorException: UNKOWN_ERROR: Cannot run
> program "scp": error=2, No such file or directory
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:599)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.start(SshActionExecutor.java:204)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:228)
> at
> org.apache.oozie.command.wf.ActionStartXCommand.execute(ActionStartXCommand.java:63)
> at org.apache.oozie.command.XCommand.call(XCommand.java:281)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:323)
> at
> org.apache.oozie.service.CallableQueueService$CompositeCallable.call(CallableQueueService.java:252)
> at
> org.apache.oozie.service.CallableQueueService$CallableWrapper.run(CallableQueueService.java:174)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.io.IOException: Cannot run program "scp": error=2, No such
> file or directory
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1041)
> at java.lang.Runtime.exec(Runtime.java:617)
> at java.lang.Runtime.exec(Runtime.java:485)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.executeCommand(SshActionExecutor.java:332)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.setupRemote(SshActionExecutor.java:376)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:206)
> at
> org.apache.oozie.action.ssh.SshActionExecutor$1.call(SshActionExecutor.java:204)
> at
> org.apache.oozie.action.ssh.SshActionExecutor.execute(SshActionExecutor.java:548)
> ... 10 more
> Caused by: java.io.IOException: error=2, No such file or directory
> at java.lang.UNIXProcess.forkAndExec(Native Method)
> at java.lang.UNIXProcess.(UNIXProcess.java:135)
> at java.lang.ProcessImpl.start(ProcessImpl.java:130)
> at java.lang.ProcessBuilder.start(ProcessBuilder.java:1022)
> ... 17 more
> {noformat}
> I checked the codes in {{org.apache.oozie.action.ssh.SshActionExecutor}}
> class, and I found oozie would execute 'scp' command to copy two
> files(ssh-base.sh,ssh-wrapper.sh) to the workspace folder on remote host. But
> the logs could not show any information about which file is not found.
> The workspace folder on remote host was created so the 'ssh'
[jira] [Commented] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936746#comment-16936746 ] Hadoop QA commented on OOZIE-3542: -- PreCommit-OOZIE-Build started > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy updated OOZIE-3542: -- Attachment: OOZIE-3542-amend-01.patch > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch, OOZIE-3542-amend-01.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Reopened] (OOZIE-3542) Handle better old Hdfs implementations in ECPolicyDisabler
[ https://issues.apache.org/jira/browse/OOZIE-3542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zsombor Gegesy reopened OOZIE-3542: --- > Handle better old Hdfs implementations in ECPolicyDisabler > -- > > Key: OOZIE-3542 > URL: https://issues.apache.org/jira/browse/OOZIE-3542 > Project: Oozie > Issue Type: Bug > Components: tools >Reporter: Zsombor Gegesy >Assignee: Zsombor Gegesy >Priority: Major > Fix For: 5.2.0 > > Attachments: OOZIE-3542-2.patch, OOZIE-3542-3.patch, > OOZIE-3542-4.patch > > > Currently, ECPolicyDisabler checks if the local hdfs implementation has the > necessary methods to get and set erasure coding policy. However, if the > namenode implementation is old, it could throw a > org.apache.hadoop.ipc.RemoteException with > RpcErrorCodeProto.ERROR_NO_SUCH_METHOD value in it. > In this case, ECPolicyDisabler fails, and prevents the installation to > succeed. > This case should be handled just like, when erasure coding is not supported. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3543) Upgrade quartz to 2.3.1
[ https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936702#comment-16936702 ] ASF subversion and git services commented on OOZIE-3543: Commit 8a4b6b5bb055fc700f819439a8e8e43cd463154f in oozie's branch refs/heads/master from Andras Salamon [ https://gitbox.apache.org/repos/asf?p=oozie.git;h=8a4b6b5 ] OOZIE-3543 Upgrade quartz to 2.3.1 (matijhs via asalamon74) > Upgrade quartz to 2.3.1 > --- > > Key: OOZIE-3543 > URL: https://issues.apache.org/jira/browse/OOZIE-3543 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-3543.patch > > > Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 > 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which > depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL > is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3543) Upgrade quartz to 2.3.1
[
https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936697#comment-16936697
]
Hadoop QA commented on OOZIE-3543:
--
Testing JIRA OOZIE-3543
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [20] new bugs found below threshold in total
that must be fixed.
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [webapp].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:red}-1{color} There are [15] new bugs found below threshold in
[tools] that must be fixed, listing only the first [5] ones.
.You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
.The top [5] most important SpotBugs errors are:
.At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
.At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
.At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
.At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
.At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
.{color:red}-1{color} There are [5] new bugs found below threshold in
[core] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The most important SpotBugs errors are:
.At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
.At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
.This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
.At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
.{color:orange}0{color} There are [4] new bugs found in [server] that would
be nice to have fixed.
.You can find the SpotBugs diff here: server/findbugs-new.html
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3181
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1222/
> Upgrade quartz to 2.3.1
> -
Failed: OOZIE-3543 PreCommit Build #1222
Jira: https://issues.apache.org/jira/browse/OOZIE-3543
Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1222/
###
## LAST 100 LINES OF THE CONSOLE
###
[...truncated 2.75 MB...]
Running test-patch task TESTS
Running test-patch task DISTRO
Running test-patch task MODERNIZER
Running modernizer op=report
[TRACE] grep -c '.*ERROR.*Prefer'
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-clean.txt
[TRACE] Modernizer errors before patch: 69
[TRACE] grep -c '.*ERROR.*Prefer'
/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-patch.txt
[TRACE] Modernizer errors after patch: 69
Testing JIRA OOZIE-3543
Cleaning local git workspace
+1 PATCH_APPLIES
+1 CLEAN
-1 RAW_PATCH_ANALYSIS
+1 the patch does not introduce any @author tags
+1 the patch does not introduce any tabs
+1 the patch does not introduce any trailing spaces
+1 the patch does not introduce any star imports
+1 the patch does not introduce any line longer than 132
-1 the patch does not add/modify any testcase
+1 RAT
+1 the patch does not seem to introduce new RAT warnings
+1 JAVADOC
+1 Javadoc generation succeeded with the patch
+1 the patch does not seem to introduce new Javadoc warning(s)
+1 COMPILE
+1 HEAD compiles
+1 patch compiles
+1 the patch does not seem to introduce new javac warnings
-1 There are [20] new bugs found below threshold in total that must be fixed.
+1 There are no new bugs found in [sharelib/oozie].
+1 There are no new bugs found in [sharelib/git].
+1 There are no new bugs found in [sharelib/sqoop].
+1 There are no new bugs found in [sharelib/pig].
+1 There are no new bugs found in [sharelib/streaming].
+1 There are no new bugs found in [sharelib/spark].
+1 There are no new bugs found in [sharelib/hcatalog].
+1 There are no new bugs found in [sharelib/hive2].
+1 There are no new bugs found in [sharelib/hive].
+1 There are no new bugs found in [sharelib/distcp].
+1 There are no new bugs found in [docs].
+1 There are no new bugs found in [examples].
+1 There are no new bugs found in [fluent-job/fluent-job-api].
+1 There are no new bugs found in [webapp].
+1 There are no new bugs found in [client].
-1 There are [15] new bugs found below threshold in [tools] that must be
fixed, listing only the first [5] ones.
You can find the SpotBugs diff here (look for the red and orange ones):
tools/findbugs-new.html
The top [5] most important SpotBugs errors are:
At OozieDBCLI.java:[line 584]: This use of
java/sql/Statement.executeUpdate(Ljava/lang/String;)I can be vulnerable to SQL
injection
At OozieDBCLI.java:[line 574]: At OozieDBCLI.java:[line 573]
At OozieDBCLI.java:[line 577]: At OozieDBCLI.java:[line 575]
At OozieDBCLI.java:[line 579]: At OozieDBCLI.java:[line 578]
At OozieDBCLI.java:[line 584]: At OozieDBCLI.java:[line 581]
-1 There are [5] new bugs found below threshold in [core] that must be
fixed.
You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
The most important SpotBugs errors are:
At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection: At BulkJPAExecutor.java:[line 206]
At BulkJPAExecutor.java:[line 111]: At BulkJPAExecutor.java:[line 127]
0 There are [4] new bugs found in [server] that would be nice to have fixed.
You can find the SpotBugs diff here: server/findbugs-new.html
+1 BACKWARDS_COMPATIBILITY
+1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient
annotations
+1 the patch does not modify JPA files
+1 TESTS
Tests run: 3181
+1 DISTRO
+1 distro tarball builds with the patch
+1 MODERNIZER
-1 Overall result, please check the reported -1(s)
The full output of the test-patch run is available at
https://builds.apache.org/job/PreCommit-OOZIE-Build/1222/
Adding comment to JIRA
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
0 00 00 0 0 0 --:--:-- --:--:-- --:--:--
0100 47710 0 100 4771 0 4026 0:00:01 0:00:01 --:--:--
4029{"self":"https://issues.apache.org/jira/rest/api/2/issue/13258484/comment/16936697","id":"16936697","
[jira] [Commented] (OOZIE-3543) Upgrade quartz to 2.3.1
[ https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936619#comment-16936619 ] Hadoop QA commented on OOZIE-3543: -- PreCommit-OOZIE-Build started > Upgrade quartz to 2.3.1 > --- > > Key: OOZIE-3543 > URL: https://issues.apache.org/jira/browse/OOZIE-3543 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-3543.patch > > > Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 > 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which > depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL > is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (OOZIE-3543) Upgrade quartz to 2.3.1
[ https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16936613#comment-16936613 ] Andras Salamon commented on OOZIE-3543: --- Thanks [~matijhs], +1, pending jenkins. > Upgrade quartz to 2.3.1 > --- > > Key: OOZIE-3543 > URL: https://issues.apache.org/jira/browse/OOZIE-3543 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-3543.patch > > > Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 > 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which > depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL > is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (OOZIE-3543) Upgrade quartz to 2.3.1
[ https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz updated OOZIE-3543: --- Attachment: OOZIE-3543.patch > Upgrade quartz to 2.3.1 > --- > > Key: OOZIE-3543 > URL: https://issues.apache.org/jira/browse/OOZIE-3543 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Blocker > Fix For: 5.2.0 > > Attachments: OOZIE-3543.patch > > > Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 > 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which > depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL > is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (OOZIE-3543) Upgrade quartz to 2.3.1
[ https://issues.apache.org/jira/browse/OOZIE-3543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mate Juhasz reassigned OOZIE-3543: -- Assignee: Mate Juhasz > Upgrade quartz to 2.3.1 > --- > > Key: OOZIE-3543 > URL: https://issues.apache.org/jira/browse/OOZIE-3543 > Project: Oozie > Issue Type: Bug >Reporter: Andras Salamon >Assignee: Mate Juhasz >Priority: Blocker > Fix For: 5.2.0 > > > Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 > 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which > depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL > is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (OOZIE-3543) Upgrade quartz to 2.3.1
Andras Salamon created OOZIE-3543: - Summary: Upgrade quartz to 2.3.1 Key: OOZIE-3543 URL: https://issues.apache.org/jira/browse/OOZIE-3543 Project: Oozie Issue Type: Bug Reporter: Andras Salamon Fix For: 5.2.0 Oozie uses quartz 2.1.7, which is six years old and depends on c3p0:c3p0 0.9.1.1 which has LGPL license. We should upgrade to quartz 2.3.1 which depends on com.mchange:c3p0. This newer version of c3p0 has dual license, EPL is good for us. -- This message was sent by Atlassian Jira (v8.3.4#803005)
