[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (93 issues) Subscriber: ooziedaily Key Summary OOZIE-3574 JavaAction create incorrect fileSystem instance in addActionLibs method https://issues.apache.org/jira/browse/OOZIE-3574 OOZIE-3569 SSH Action should add checking success file https://issues.apache.org/jira/browse/OOZIE-3569 OOZIE-3568 Have large amount of log information “WARN messages [main] openjpa.MetaData” in jetty.log need to clean https://issues.apache.org/jira/browse/OOZIE-3568 OOZIE-3567 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3567 OOZIE-3560 IDEA shows have some error in index.jsp https://issues.apache.org/jira/browse/OOZIE-3560 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3254 [coordinator] LAST_ONLY and NONE execution modes: possible OutOfMemoryError when there are too many coordinator actions to materialize https://issues.apache.org/jira/browse/OOZIE-3254 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2812 SparkConfigurationService should support loading configurations from multiple Spark versions https://issues.apache.org/jira/browse/OOZIE-2812 OOZIE-2795 Create lib directory or symlink for Oozie CLI during packaging
[jira] [Commented] (OOZIE-3569) SSH Action should add checking success file
[
https://issues.apache.org/jira/browse/OOZIE-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021782#comment-17021782
]
Hadoop QA commented on OOZIE-3569:
--
Testing JIRA OOZIE-3569
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:red}-1{color} the patch contains 3 line(s) with trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [3] new bugs found below threshold in total that
must be fixed.
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:red}-1{color} There are [3] new bugs found below threshold in
[core] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The most important SpotBugs errors are:
.At AuthorizationService.java:[line 192]:
java/io/File.(Ljava/lang/String;Ljava/lang/String;)V reads a file
whose location might be specified by user input
.At AuthorizationService.java:[line 191]: At
AuthorizationService.java:[line 189]
.At ShareLibService.java:[line 695]: Unsafe comparison of hash that are
susceptible to timing attack
.At ShareLibService.java:[line 691]: At ShareLibService.java:[line 689]
.Possible injection that can lead to Source spoofing, header override and
email body injection.: At ShareLibService.java:[line 693]
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3207
.{color:orange}Tests failed at first run:{color}
TestBlockingInputStream#testFastWritingBlockingInputStream
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1293/
> SSH Action should add checking success file
> ---
>
> Key: OOZIE-3569
> URL: https://issues.apache.org/jira/browse/OOZIE-3569
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3569-v1.patch, OOZIE-3569-v2.patch,
> OOZIE-3569-v3.patch
>
>
> *Phenomenon*
> Currently, {{SSH Action}} checking operation are as following:
> Firstly, check operation is to check {{Oozie}} ppid. When pgid does not
> exist, che
Failed: OOZIE-3569 PreCommit Build #1293
Jira: https://issues.apache.org/jira/browse/OOZIE-3569 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1293/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.15 MB...] [TRACE] Summary file size is 3323 bytes [TRACE] Full summary file size is 2577 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Running test-patch task MODERNIZER Running modernizer op=report [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-clean.txt [TRACE] Modernizer errors before patch: 52 [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-patch.txt [TRACE] Modernizer errors after patch: 52 Testing JIRA OOZIE-3569 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs -1 the patch contains 3 line(s) with trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 -1 the patch does not add/modify any testcase +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [3] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [fluent-job/fluent-job-api]. +1 There are no new bugs found in [docs]. -1 There are [3] new bugs found below threshold in [core] that must be fixed. You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html The most important SpotBugs errors are: At AuthorizationService.java:[line 192]: java/io/File.(Ljava/lang/String;Ljava/lang/String;)V reads a file whose location might be specified by user input At AuthorizationService.java:[line 191]: At AuthorizationService.java:[line 189] At ShareLibService.java:[line 695]: Unsafe comparison of hash that are susceptible to timing attack At ShareLibService.java:[line 691]: At ShareLibService.java:[line 689] Possible injection that can lead to Source spoofing, header override and email body injection.: At ShareLibService.java:[line 693] +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [webapp]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 3207 Tests failed at first run: TestBlockingInputStream#testFastWritingBlockingInputStream For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch +1 MODERNIZER -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1293/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 42070 0 100 4207 0 2819 0:00:01 0:00:01 --:--:-- 2819{"self":"https://issues.apache.org/jira/rest/api/2/issue/13274936/comment/17021782","id":"17021782","author":{"self":"https://issues.apache.org/jira/rest/api/2/user?username=hadoopqa","name":"hadoopqa","key":"hadoopqa","avatarUrls":{"48x48":"https://
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021760#comment-17021760
]
Hadoop QA commented on OOZIE-3574:
--
Testing JIRA OOZIE-3574
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Failed: OOZIE-3574 PreCommit Build #1295
Jira: https://issues.apache.org/jira/browse/OOZIE-3574 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1295/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 901.94 KB...] [INFO] [INFO] --< org.apache.oozie:oozie-zookeeper-security-tests >--- [INFO] Building Apache Oozie ZooKeeper Security Tests 5.3.0-SNAPSHOT[25/25] [INFO] [ jar ]- [INFO] [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ oozie-zookeeper-security-tests --- [INFO] Deleting /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/target [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ oozie-zookeeper-security-tests --- [INFO] [INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ oozie-zookeeper-security-tests --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ oozie-zookeeper-security-tests --- [INFO] No sources to compile [INFO] [INFO] --- modernizer-maven-plugin:1.8.0:modernizer (default-cli) @ oozie-zookeeper-security-tests --- [INFO] [INFO] Reactor Summary for Apache Oozie Main 5.3.0-SNAPSHOT: [INFO] [INFO] Apache Oozie Main .. SUCCESS [ 0.957 s] [INFO] Apache Oozie Fluent Job SUCCESS [ 0.091 s] [INFO] Apache Oozie Fluent Job API SUCCESS [ 6.577 s] [INFO] Apache Oozie Client SUCCESS [ 1.681 s] [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 1.245 s] [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ 0.965 s] [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ 0.333 s] [INFO] Apache Oozie Core .. SUCCESS [ 8.213 s] [INFO] Apache Oozie Share Lib Streaming ... SUCCESS [ 1.626 s] [INFO] Apache Oozie Share Lib Pig . SUCCESS [ 1.229 s] [INFO] Apache Oozie Share Lib Git . SUCCESS [ 1.868 s] [INFO] Apache Oozie Share Lib Hive SUCCESS [ 0.981 s] [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [ 1.483 s] [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [ 1.588 s] [INFO] Apache Oozie Examples .. SUCCESS [ 1.577 s] [INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 2.117 s] [INFO] Apache Oozie Share Lib . SUCCESS [ 0.015 s] [INFO] Apache Oozie Docs .. SUCCESS [ 0.032 s] [INFO] Apache Oozie WebApp SUCCESS [ 1.810 s] [INFO] Apache Oozie Tools . SUCCESS [ 1.407 s] [INFO] Apache Oozie MiniOozie . SUCCESS [ 1.392 s] [INFO] Apache Oozie Fluent Job Client . SUCCESS [ 0.671 s] [INFO] Apache Oozie Server SUCCESS [ 2.189 s] [INFO] Apache Oozie Distro SUCCESS [ 1.742 s] [INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [ 1.617 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 44.939 s [INFO] Finished at: 2020-01-23T04:18:41Z [INFO] EXITVALUE 0 [TRACE] Modernizer output in HEAD verified and saved Applying patch Checking patch a/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java => b/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java... error: a/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java: No such file or directory Checking patch a/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java => b/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java... error: a/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java: No such file or directory Checking patch a/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java => b/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java... error: a/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java: No such file or directory Checking p
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021754#comment-17021754
]
Hadoop QA commented on OOZIE-3574:
--
PreCommit-OOZIE-Build started
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junfan Zhang updated OOZIE-3574:
Attachment: OOZIE-3574-v3.patch
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junfan Zhang updated OOZIE-3574:
Attachment: (was: OOZIE-3574-v3.patch)
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021749#comment-17021749
]
Hadoop QA commented on OOZIE-3574:
--
Testing JIRA OOZIE-3574
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Failed: OOZIE-3574 PreCommit Build #1294
Jira: https://issues.apache.org/jira/browse/OOZIE-3574 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1294/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 902.80 KB...] [INFO] [INFO] --< org.apache.oozie:oozie-zookeeper-security-tests >--- [INFO] Building Apache Oozie ZooKeeper Security Tests 5.3.0-SNAPSHOT[25/25] [INFO] [ jar ]- [INFO] [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ oozie-zookeeper-security-tests --- [INFO] Deleting /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/target [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ oozie-zookeeper-security-tests --- [INFO] [INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ oozie-zookeeper-security-tests --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ oozie-zookeeper-security-tests --- [INFO] No sources to compile [INFO] [INFO] --- modernizer-maven-plugin:1.8.0:modernizer (default-cli) @ oozie-zookeeper-security-tests --- [INFO] [INFO] Reactor Summary for Apache Oozie Main 5.3.0-SNAPSHOT: [INFO] [INFO] Apache Oozie Main .. SUCCESS [ 0.985 s] [INFO] Apache Oozie Fluent Job SUCCESS [ 0.071 s] [INFO] Apache Oozie Fluent Job API SUCCESS [ 6.699 s] [INFO] Apache Oozie Client SUCCESS [ 1.581 s] [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 1.237 s] [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ 0.937 s] [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ 0.246 s] [INFO] Apache Oozie Core .. SUCCESS [ 8.911 s] [INFO] Apache Oozie Share Lib Streaming ... SUCCESS [ 2.215 s] [INFO] Apache Oozie Share Lib Pig . SUCCESS [ 1.341 s] [INFO] Apache Oozie Share Lib Git . SUCCESS [ 1.957 s] [INFO] Apache Oozie Share Lib Hive SUCCESS [ 0.889 s] [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [ 1.672 s] [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [ 1.687 s] [INFO] Apache Oozie Examples .. SUCCESS [ 1.756 s] [INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 2.174 s] [INFO] Apache Oozie Share Lib . SUCCESS [ 0.015 s] [INFO] Apache Oozie Docs .. SUCCESS [ 0.032 s] [INFO] Apache Oozie WebApp SUCCESS [ 1.755 s] [INFO] Apache Oozie Tools . SUCCESS [ 1.484 s] [INFO] Apache Oozie MiniOozie . SUCCESS [ 1.604 s] [INFO] Apache Oozie Fluent Job Client . SUCCESS [ 1.128 s] [INFO] Apache Oozie Server SUCCESS [ 2.009 s] [INFO] Apache Oozie Distro SUCCESS [ 1.744 s] [INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [ 1.732 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 47.635 s [INFO] Finished at: 2020-01-23T03:59:44Z [INFO] EXITVALUE 0 [TRACE] Modernizer output in HEAD verified and saved Applying patch Checking patch a/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java => b/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java... error: a/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java: No such file or directory Checking patch a/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java => b/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java... error: a/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java: No such file or directory Checking patch a/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java => b/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java... error: a/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java: No such file or directory Checking p
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021738#comment-17021738
]
Hadoop QA commented on OOZIE-3574:
--
PreCommit-OOZIE-Build started
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021731#comment-17021731
]
Junfan Zhang commented on OOZIE-3574:
-
[~asalamon74] v3 patch was uploaded. I remove some unnecessary code and avoid
duplicate calls.
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junfan Zhang updated OOZIE-3574:
Attachment: OOZIE-3574-v3.patch
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch,
> OOZIE-3574-v3.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3569) SSH Action should add checking success file
[
https://issues.apache.org/jira/browse/OOZIE-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021727#comment-17021727
]
Junfan Zhang commented on OOZIE-3569:
-
[~asalamon74] Of course, we use Oozie on a large scale in our production
environment and accumulate some practice and experience. I'm pleased to
contribute to the community.
Closer to home, v3-patch was uploaded.
> SSH Action should add checking success file
> ---
>
> Key: OOZIE-3569
> URL: https://issues.apache.org/jira/browse/OOZIE-3569
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3569-v1.patch, OOZIE-3569-v2.patch,
> OOZIE-3569-v3.patch
>
>
> *Phenomenon*
> Currently, {{SSH Action}} checking operation are as following:
> Firstly, check operation is to check {{Oozie}} ppid. When pgid does not
> exist, check whether there is an error file. If the error file does not
> exist, {{Oozie}} will set action status {{OK}}
> However, when {{Oozie}} pgid is killed externally, this action will be
> incorrectly determined to be successful.
> *Solution*
> In ssh-wrapper.sh, when command execution is OK, {{Oozie}} should touch a
> success empty file like touching error file.
> In {{SshActionExecutor}} check method, Oozie should add checking the success
> file existence.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3569) SSH Action should add checking success file
[
https://issues.apache.org/jira/browse/OOZIE-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021725#comment-17021725
]
Hadoop QA commented on OOZIE-3569:
--
PreCommit-OOZIE-Build started
> SSH Action should add checking success file
> ---
>
> Key: OOZIE-3569
> URL: https://issues.apache.org/jira/browse/OOZIE-3569
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3569-v1.patch, OOZIE-3569-v2.patch,
> OOZIE-3569-v3.patch
>
>
> *Phenomenon*
> Currently, {{SSH Action}} checking operation are as following:
> Firstly, check operation is to check {{Oozie}} ppid. When pgid does not
> exist, check whether there is an error file. If the error file does not
> exist, {{Oozie}} will set action status {{OK}}
> However, when {{Oozie}} pgid is killed externally, this action will be
> incorrectly determined to be successful.
> *Solution*
> In ssh-wrapper.sh, when command execution is OK, {{Oozie}} should touch a
> success empty file like touching error file.
> In {{SshActionExecutor}} check method, Oozie should add checking the success
> file existence.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3569) SSH Action should add checking success file
[
https://issues.apache.org/jira/browse/OOZIE-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junfan Zhang updated OOZIE-3569:
Attachment: OOZIE-3569-v3.patch
> SSH Action should add checking success file
> ---
>
> Key: OOZIE-3569
> URL: https://issues.apache.org/jira/browse/OOZIE-3569
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3569-v1.patch, OOZIE-3569-v2.patch,
> OOZIE-3569-v3.patch
>
>
> *Phenomenon*
> Currently, {{SSH Action}} checking operation are as following:
> Firstly, check operation is to check {{Oozie}} ppid. When pgid does not
> exist, check whether there is an error file. If the error file does not
> exist, {{Oozie}} will set action status {{OK}}
> However, when {{Oozie}} pgid is killed externally, this action will be
> incorrectly determined to be successful.
> *Solution*
> In ssh-wrapper.sh, when command execution is OK, {{Oozie}} should touch a
> success empty file like touching error file.
> In {{SshActionExecutor}} check method, Oozie should add checking the success
> file existence.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021230#comment-17021230
]
Andras Salamon commented on OOZIE-3574:
---
[~zuston] Thanks for adding the unit test, I wanted to suggest a cleanup, but
it seems to me you already removed the dead code. Would it be possible to call
{{getActionLibPathWithFs2}} from {{createTestActionLibPathsWithFs2}} and avoid
some code duplication?
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3569) SSH Action should add checking success file
[
https://issues.apache.org/jira/browse/OOZIE-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021219#comment-17021219
]
Hadoop QA commented on OOZIE-3569:
--
Testing JIRA OOZIE-3569
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:green}+1{color} the patch does not introduce any line longer than
132
.{color:red}-1{color} the patch does not add/modify any testcase
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [4] new bugs found below threshold in total that
must be fixed.
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:red}-1{color} There are [3] new bugs found below threshold in
[core] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The most important SpotBugs errors are:
.At AuthorizationService.java:[line 192]:
java/io/File.(Ljava/lang/String;Ljava/lang/String;)V reads a file
whose location might be specified by user input
.At AuthorizationService.java:[line 191]: At
AuthorizationService.java:[line 189]
.At ShareLibService.java:[line 695]: Unsafe comparison of hash that are
susceptible to timing attack
.At ShareLibService.java:[line 691]: At ShareLibService.java:[line 689]
.Possible injection that can lead to Source spoofing, header override and
email body injection.: At ShareLibService.java:[line 693]
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:red}-1{color} There are [1] new bugs found below threshold in
[sharelib/oozie] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
sharelib/oozie/findbugs-new.html
.The most important SpotBugs errors are:
.At ShellMain.java:[line 93]: This usage of
java/lang/ProcessBuilder.(Ljava/util/List;)V can be vulnerable to
Command Injection
.At ShellMain.java:[line 91]: At ShellMain.java:[line 90]
.At ShellMain.java:[line 92]
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3207
.{color:orange}Tests failed at first run:{color}
TestBlockingInputStream#testFastWritingBlockingInputStream
TestBlockingInputStream#testLimitedWritingBlockingInputStream
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1291/
> SSH Action should add checking success file
> ---
>
> Key: OOZIE-3569
>
Failed: OOZIE-3569 PreCommit Build #1291
Jira: https://issues.apache.org/jira/browse/OOZIE-3569 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1291/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.16 MB...] Running test-patch task DISTRO Running test-patch task MODERNIZER Running modernizer op=report [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-clean.txt [TRACE] Modernizer errors before patch: 52 [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-patch.txt [TRACE] Modernizer errors after patch: 52 Testing JIRA OOZIE-3569 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports +1 the patch does not introduce any line longer than 132 -1 the patch does not add/modify any testcase +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [4] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [fluent-job/fluent-job-api]. +1 There are no new bugs found in [docs]. -1 There are [3] new bugs found below threshold in [core] that must be fixed. You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html The most important SpotBugs errors are: At AuthorizationService.java:[line 192]: java/io/File.(Ljava/lang/String;Ljava/lang/String;)V reads a file whose location might be specified by user input At AuthorizationService.java:[line 191]: At AuthorizationService.java:[line 189] At ShareLibService.java:[line 695]: Unsafe comparison of hash that are susceptible to timing attack At ShareLibService.java:[line 691]: At ShareLibService.java:[line 689] Possible injection that can lead to Source spoofing, header override and email body injection.: At ShareLibService.java:[line 693] +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/pig]. -1 There are [1] new bugs found below threshold in [sharelib/oozie] that must be fixed. You can find the SpotBugs diff here (look for the red and orange ones): sharelib/oozie/findbugs-new.html The most important SpotBugs errors are: At ShellMain.java:[line 93]: This usage of java/lang/ProcessBuilder. (Ljava/util/List;)V can be vulnerable to Command Injection At ShellMain.java:[line 91]: At ShellMain.java:[line 90] At ShellMain.java:[line 92] +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [webapp]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 3207 Tests failed at first run: TestBlockingInputStream#testFastWritingBlockingInputStream TestBlockingInputStream#testLimitedWritingBlockingInputStream For the complete list of flaky tests, see TEST-SUMMARY-FULL files. +1 DISTRO +1 distro tarball builds with the patch +1 MODERNIZER -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1291/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 47040 0 100 4704 0 3176 0:00:01 0:00:01 --:--:-- 3176{"self":"https://issues.apache.org/jira/rest/api/2/issue/13274936/comment/17021219","id":"17021219","author":{"self":"https://issues.apache.org/jira/res
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021076#comment-17021076
]
Hadoop QA commented on OOZIE-3574:
--
Testing JIRA OOZIE-3574
Cleaning local git workspace
{color:red}-1{color} Patch failed to apply to head of branch
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Failed: OOZIE-3574 PreCommit Build #1292
Jira: https://issues.apache.org/jira/browse/OOZIE-3574 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1292/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 902.74 KB...] [INFO] [INFO] --< org.apache.oozie:oozie-zookeeper-security-tests >--- [INFO] Building Apache Oozie ZooKeeper Security Tests 5.3.0-SNAPSHOT[25/25] [INFO] [ jar ]- [INFO] [INFO] --- maven-clean-plugin:3.0.0:clean (default-clean) @ oozie-zookeeper-security-tests --- [INFO] Deleting /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/target [INFO] [INFO] --- maven-remote-resources-plugin:1.5:process (process-resource-bundles) @ oozie-zookeeper-security-tests --- [INFO] [INFO] --- maven-resources-plugin:3.0.2:resources (default-resources) @ oozie-zookeeper-security-tests --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/zookeeper-security-tests/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ oozie-zookeeper-security-tests --- [INFO] No sources to compile [INFO] [INFO] --- modernizer-maven-plugin:1.8.0:modernizer (default-cli) @ oozie-zookeeper-security-tests --- [INFO] [INFO] Reactor Summary for Apache Oozie Main 5.3.0-SNAPSHOT: [INFO] [INFO] Apache Oozie Main .. SUCCESS [ 0.901 s] [INFO] Apache Oozie Fluent Job SUCCESS [ 0.074 s] [INFO] Apache Oozie Fluent Job API SUCCESS [ 6.832 s] [INFO] Apache Oozie Client SUCCESS [ 1.649 s] [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ 1.241 s] [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ 0.928 s] [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ 0.262 s] [INFO] Apache Oozie Core .. SUCCESS [ 8.896 s] [INFO] Apache Oozie Share Lib Streaming ... SUCCESS [ 1.995 s] [INFO] Apache Oozie Share Lib Pig . SUCCESS [ 1.307 s] [INFO] Apache Oozie Share Lib Git . SUCCESS [ 1.974 s] [INFO] Apache Oozie Share Lib Hive SUCCESS [ 1.104 s] [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [ 1.587 s] [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [ 1.961 s] [INFO] Apache Oozie Examples .. SUCCESS [ 1.805 s] [INFO] Apache Oozie Share Lib Spark ... SUCCESS [ 2.185 s] [INFO] Apache Oozie Share Lib . SUCCESS [ 0.015 s] [INFO] Apache Oozie Docs .. SUCCESS [ 0.031 s] [INFO] Apache Oozie WebApp SUCCESS [ 1.829 s] [INFO] Apache Oozie Tools . SUCCESS [ 1.203 s] [INFO] Apache Oozie MiniOozie . SUCCESS [ 1.239 s] [INFO] Apache Oozie Fluent Job Client . SUCCESS [ 1.091 s] [INFO] Apache Oozie Server SUCCESS [ 1.985 s] [INFO] Apache Oozie Distro SUCCESS [ 1.878 s] [INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [ 1.658 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 47.396 s [INFO] Finished at: 2020-01-22T13:54:13Z [INFO] EXITVALUE 0 [TRACE] Modernizer output in HEAD verified and saved Applying patch Checking patch a/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java => b/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java... error: a/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java: No such file or directory Checking patch a/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java => b/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java... error: a/core/src/test/java/org/apache/oozie/action/hadoop/ActionExecutorTestCase.java: No such file or directory Checking patch a/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java => b/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java... error: a/core/src/test/java/org/apache/oozie/action/hadoop/TestJavaActionExecutorLibAddition.java: No such file or directory Checking p
[jira] [Commented] (OOZIE-3574) JavaAction create incorrect fileSystem instance in addActionLibs method
[
https://issues.apache.org/jira/browse/OOZIE-3574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021064#comment-17021064
]
Hadoop QA commented on OOZIE-3574:
--
PreCommit-OOZIE-Build started
> JavaAction create incorrect fileSystem instance in addActionLibs method
> ---
>
> Key: OOZIE-3574
> URL: https://issues.apache.org/jira/browse/OOZIE-3574
> Project: Oozie
> Issue Type: Sub-task
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3574-v1.patch, OOZIE-3574-v2.patch
>
>
> Code is
> [here|https://github.com/apache/oozie/blob/9c288fe5cea6f2fbbae76f720b9e215acdd07709/core/src/main/java/org/apache/oozie/action/hadoop/JavaActionExecutor.java#L734].
> If actionlibPath scheme is different from appPath (like actionLibPath's
> scheme is s3a, but the appPath is hdfs), this will fail to execute
> {{fs.exist(actionLibsPath)}}. So i think Oozie should create fileSystem by
> actionLibsPath.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3569) SSH Action should add checking success file
[
https://issues.apache.org/jira/browse/OOZIE-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021065#comment-17021065
]
Hadoop QA commented on OOZIE-3569:
--
PreCommit-OOZIE-Build started
> SSH Action should add checking success file
> ---
>
> Key: OOZIE-3569
> URL: https://issues.apache.org/jira/browse/OOZIE-3569
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3569-v1.patch, OOZIE-3569-v2.patch
>
>
> *Phenomenon*
> Currently, {{SSH Action}} checking operation are as following:
> Firstly, check operation is to check {{Oozie}} ppid. When pgid does not
> exist, check whether there is an error file. If the error file does not
> exist, {{Oozie}} will set action status {{OK}}
> However, when {{Oozie}} pgid is killed externally, this action will be
> incorrectly determined to be successful.
> *Solution*
> In ssh-wrapper.sh, when command execution is OK, {{Oozie}} should touch a
> success empty file like touching error file.
> In {{SshActionExecutor}} check method, Oozie should add checking the success
> file existence.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3569) SSH Action should add checking success file
[
https://issues.apache.org/jira/browse/OOZIE-3569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17020919#comment-17020919
]
Andras Salamon commented on OOZIE-3569:
---
Yes [~zuston] I understand the upgrade difficulty, that's why I asked
[~gezapeti].
I think his suggestion is good. This is a very useful patch, we should only
document this difficulty.
Could you please modify the documentation and add a small section which
describes this files and also point out that Oozie 5.3.0 introduced the new
file which might cause problems during update.
> SSH Action should add checking success file
> ---
>
> Key: OOZIE-3569
> URL: https://issues.apache.org/jira/browse/OOZIE-3569
> Project: Oozie
> Issue Type: Bug
>Reporter: Junfan Zhang
>Assignee: Junfan Zhang
>Priority: Major
> Attachments: OOZIE-3569-v1.patch, OOZIE-3569-v2.patch
>
>
> *Phenomenon*
> Currently, {{SSH Action}} checking operation are as following:
> Firstly, check operation is to check {{Oozie}} ppid. When pgid does not
> exist, check whether there is an error file. If the error file does not
> exist, {{Oozie}} will set action status {{OK}}
> However, when {{Oozie}} pgid is killed externally, this action will be
> incorrectly determined to be successful.
> *Solution*
> In ssh-wrapper.sh, when command execution is OK, {{Oozie}} should touch a
> success empty file like touching error file.
> In {{SshActionExecutor}} check method, Oozie should add checking the success
> file existence.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
