[jira] Subscription: Oozie Patch Available
Issue Subscription Filter: Oozie Patch Available (95 issues) Subscriber: ooziedaily Key Summary OOZIE-3561 Forkjoin validation is slow when there are many actions in chain https://issues.apache.org/jira/browse/OOZIE-3561 OOZIE-3560 IDEA shows have some error in index.jsp https://issues.apache.org/jira/browse/OOZIE-3560 OOZIE-3557 [build] Update required minimum maven version https://issues.apache.org/jira/browse/OOZIE-3557 OOZIE-3549 Add back support for truststore passwords https://issues.apache.org/jira/browse/OOZIE-3549 OOZIE-3538 oozie-main(pom.xml) plugin maven-surefire-plugin configuration the Tag value is wrong https://issues.apache.org/jira/browse/OOZIE-3538 OOZIE-3491 Confusing System ID error message https://issues.apache.org/jira/browse/OOZIE-3491 OOZIE-3482 Fix bug in CoordSubmitXCommand#validateCoordinatorJob https://issues.apache.org/jira/browse/OOZIE-3482 OOZIE-3480 Add windowactionstatus metrics in DBLiteWorkflowStoreService https://issues.apache.org/jira/browse/OOZIE-3480 OOZIE-3461 CoordMaterializeTriggerService code cleanup https://issues.apache.org/jira/browse/OOZIE-3461 OOZIE-3449 Make spark-2 as the default profile https://issues.apache.org/jira/browse/OOZIE-3449 OOZIE-3447 Run test case in local : It shows oozie-hsqldb-orm.xml exception https://issues.apache.org/jira/browse/OOZIE-3447 OOZIE-3418 Upgrade to Guava 27 https://issues.apache.org/jira/browse/OOZIE-3418 OOZIE-3404 The env variable of SPARK_HOME needs to be set when running pySpark https://issues.apache.org/jira/browse/OOZIE-3404 OOZIE-3375 Can't use empty in coordinator https://issues.apache.org/jira/browse/OOZIE-3375 OOZIE-3367 Using && in EL expressions in oozie bundle.xml files generates parse errors https://issues.apache.org/jira/browse/OOZIE-3367 OOZIE-3366 Update workflow status and subworkflow status on suspend command https://issues.apache.org/jira/browse/OOZIE-3366 OOZIE-3364 Rerunning Oozie bundle jobs starts the coordinators in indeterminate order https://issues.apache.org/jira/browse/OOZIE-3364 OOZIE-3362 When killed, SSH action should kill the spawned processes on target host https://issues.apache.org/jira/browse/OOZIE-3362 OOZIE-3335 Cleanup parseFilter methods https://issues.apache.org/jira/browse/OOZIE-3335 OOZIE-3328 Create Hive compatibility action executor to run hive actions using beeline https://issues.apache.org/jira/browse/OOZIE-3328 OOZIE-3320 Oozie ShellAction should support absolute bash file path https://issues.apache.org/jira/browse/OOZIE-3320 OOZIE-3319 Log SSH action callback error output https://issues.apache.org/jira/browse/OOZIE-3319 OOZIE-3301 Update NOTICE file https://issues.apache.org/jira/browse/OOZIE-3301 OOZIE-3274 Remove slf4j https://issues.apache.org/jira/browse/OOZIE-3274 OOZIE-3266 Coord action rerun support RERUN_SKIP_NODES option https://issues.apache.org/jira/browse/OOZIE-3266 OOZIE-3256 refactor OozieCLI class https://issues.apache.org/jira/browse/OOZIE-3256 OOZIE-3254 [coordinator] LAST_ONLY and NONE execution modes: possible OutOfMemoryError when there are too many coordinator actions to materialize https://issues.apache.org/jira/browse/OOZIE-3254 OOZIE-3199 Let system property restriction configurable https://issues.apache.org/jira/browse/OOZIE-3199 OOZIE-3196 Authorization: restrict world readability by user https://issues.apache.org/jira/browse/OOZIE-3196 OOZIE-3170 Oozie Diagnostic Bundle tool fails with NPE due to missing service class https://issues.apache.org/jira/browse/OOZIE-3170 OOZIE-3137 Add support for log4j2 in HiveMain https://issues.apache.org/jira/browse/OOZIE-3137 OOZIE-3135 Configure log4j2 in SqoopMain https://issues.apache.org/jira/browse/OOZIE-3135 OOZIE-3091 Oozie Sqoop Avro Import fails with "java.lang.NoClassDefFoundError: org/apache/avro/mapred/AvroWrapper" https://issues.apache.org/jira/browse/OOZIE-3091 OOZIE-3071 Oozie 4.3 Spark sharelib ueses a different version of commons-lang3 than Spark 2.2.0 https://issues.apache.org/jira/browse/OOZIE-3071 OOZIE-3063 Sanitizing variables that are part of openjpa.ConnectionProperties https://issues.apache.org/jira/browse/OOZIE-3063 OOZIE-3062 Set HADOOP_CONF_DIR for spark action https://issues.apache.org/jira/browse/OOZIE-3062 OOZIE-2952 Fix Findbugs warnings in oozie-sharelib-oozie https://issues.apache.org/jira/browse/OOZIE-2952 OOZIE-2834 ParameterVerifier logging non-useful warning for workflow definition https://issues.apache.org/jira/browse/OOZIE-2834 OOZIE-2812 SparkConfigurationService should
[jira] [Commented] (OOZIE-3561) Forkjoin validation is slow when there are many actions in chain
[
https://issues.apache.org/jira/browse/OOZIE-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981562#comment-16981562
]
Hadoop QA commented on OOZIE-3561:
--
Testing JIRA OOZIE-3561
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:red}-1{color} the patch contains 5 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 1 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:red}-1{color} There are [4] new bugs found below threshold in total that
must be fixed.
.{color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
.{color:green}+1{color} There are no new bugs found in [docs].
.{color:red}-1{color} There are [4] new bugs found below threshold in
[core] that must be fixed.
.You can find the SpotBugs diff here (look for the red and orange ones):
core/findbugs-new.html
.The most important SpotBugs errors are:
.At BulkJPAExecutor.java:[line 206]: This use of
javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query;
can be vulnerable to SQL/JPQL injection
.At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175]
.At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199]
.java/io/File.(Ljava/lang/String;Ljava/lang/String;)V reads a
file whose location might be specified by user input: At
BulkJPAExecutor.java:[line 206]
.At AuthorizationService.java:[line 189]: At
AuthorizationService.java:[line 192]
.{color:green}+1{color} There are no new bugs found in [sharelib/spark].
.{color:green}+1{color} There are no new bugs found in [sharelib/git].
.{color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive2].
.{color:green}+1{color} There are no new bugs found in [sharelib/streaming].
.{color:green}+1{color} There are no new bugs found in [sharelib/pig].
.{color:green}+1{color} There are no new bugs found in [sharelib/oozie].
.{color:green}+1{color} There are no new bugs found in [sharelib/hive].
.{color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
.{color:green}+1{color} There are no new bugs found in [sharelib/distcp].
.{color:green}+1{color} There are no new bugs found in [tools].
.{color:green}+1{color} There are no new bugs found in [server].
.{color:green}+1{color} There are no new bugs found in [client].
.{color:green}+1{color} There are no new bugs found in [examples].
.{color:green}+1{color} There are no new bugs found in [webapp].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:green}+1 TESTS{color}
.Tests run: 3196
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:green}+1 MODERNIZER{color}
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/1251/
> Forkjoin validation is slow when there are many actions in chain
>
>
> Key: OOZIE-3561
> URL: https://issues.apache.org/jira/browse/OOZIE-3561
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: performance
> Attachments: OOZIE-3561-002.patch, OOZIE-3561_001.patch
>
>
> In case we have a workflow which has, let's say, 80 actions after each other:
> {{a1 -> a2 -> ... a80}}
> then the validator code "never" finishes.
> Currently the validation (in my understanding) does depth first checks from
> t
Failed: OOZIE-3561 PreCommit Build #1251
Jira: https://issues.apache.org/jira/browse/OOZIE-3561 Build: https://builds.apache.org/job/PreCommit-OOZIE-Build/1251/ ### ## LAST 100 LINES OF THE CONSOLE ### [...truncated 2.15 MB...] [DEBUG] There are no new bugs found in [webapp]. [ERROR] There are [4] new bugs found below threshold in total that must be fixed. [TRACE] SpotBugs diffs checked and reports created [TRACE] Summary file size is bytes [TRACE] Full summary file size is 2951 bytes [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar] removed [TRACE] File [/home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/tmp/SPOTBUGS_DIFF/diff/findbugs-diff-0.1.0-all.jar.md5] removed Running test-patch task BACKWARDS_COMPATIBILITY Running test-patch task TESTS Running test-patch task DISTRO Running test-patch task MODERNIZER Running modernizer op=report [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-clean.txt [TRACE] Modernizer errors before patch: 51 [TRACE] grep -c '.*ERROR.*Prefer' /home/jenkins/jenkins-slave/workspace/PreCommit-OOZIE-Build/test-patch/reports/MODERNIZER-patch.txt [TRACE] Modernizer errors after patch: 51 Testing JIRA OOZIE-3561 Cleaning local git workspace +1 PATCH_APPLIES +1 CLEAN -1 RAW_PATCH_ANALYSIS +1 the patch does not introduce any @author tags +1 the patch does not introduce any tabs +1 the patch does not introduce any trailing spaces +1 the patch does not introduce any star imports -1 the patch contains 5 line(s) longer than 132 characters +1 the patch adds/modifies 1 testcase(s) +1 RAT +1 the patch does not seem to introduce new RAT warnings +1 JAVADOC +1 Javadoc generation succeeded with the patch +1 the patch does not seem to introduce new Javadoc warning(s) +1 COMPILE +1 HEAD compiles +1 patch compiles +1 the patch does not seem to introduce new javac warnings -1 There are [4] new bugs found below threshold in total that must be fixed. +1 There are no new bugs found in [fluent-job/fluent-job-api]. +1 There are no new bugs found in [docs]. -1 There are [4] new bugs found below threshold in [core] that must be fixed. You can find the SpotBugs diff here (look for the red and orange ones): core/findbugs-new.html The most important SpotBugs errors are: At BulkJPAExecutor.java:[line 206]: This use of javax/persistence/EntityManager.createQuery(Ljava/lang/String;)Ljavax/persistence/Query; can be vulnerable to SQL/JPQL injection At BulkJPAExecutor.java:[line 176]: At BulkJPAExecutor.java:[line 175] At BulkJPAExecutor.java:[line 205]: At BulkJPAExecutor.java:[line 199] java/io/File.(Ljava/lang/String;Ljava/lang/String;)V reads a file whose location might be specified by user input: At BulkJPAExecutor.java:[line 206] At AuthorizationService.java:[line 189]: At AuthorizationService.java:[line 192] +1 There are no new bugs found in [sharelib/spark]. +1 There are no new bugs found in [sharelib/git]. +1 There are no new bugs found in [sharelib/sqoop]. +1 There are no new bugs found in [sharelib/hive2]. +1 There are no new bugs found in [sharelib/streaming]. +1 There are no new bugs found in [sharelib/pig]. +1 There are no new bugs found in [sharelib/oozie]. +1 There are no new bugs found in [sharelib/hive]. +1 There are no new bugs found in [sharelib/hcatalog]. +1 There are no new bugs found in [sharelib/distcp]. +1 There are no new bugs found in [tools]. +1 There are no new bugs found in [server]. +1 There are no new bugs found in [client]. +1 There are no new bugs found in [examples]. +1 There are no new bugs found in [webapp]. +1 BACKWARDS_COMPATIBILITY +1 the patch does not change any JPA Entity/Colum/Basic/Lob/Transient annotations +1 the patch does not modify JPA files +1 TESTS Tests run: 3196 +1 DISTRO +1 distro tarball builds with the patch +1 MODERNIZER -1 Overall result, please check the reported -1(s) The full output of the test-patch run is available at https://builds.apache.org/job/PreCommit-OOZIE-Build/1251/ Adding comment to JIRA % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 00 00 0 0 0 --:--:-- --:--:-- --:--:-- 0100 40300 0 100 4030 0 2670 0:00:01 0:00:01 --:--:-- 2668{"self":"https://issues.apache.org/jira/rest/api/2/issue/13269851/comment/16981562","id":"16981562","author":{"self":"https://issues.apache.o
[jira] [Commented] (OOZIE-3561) Forkjoin validation is slow when there are many actions in chain
[
https://issues.apache.org/jira/browse/OOZIE-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981482#comment-16981482
]
Hadoop QA commented on OOZIE-3561:
--
PreCommit-OOZIE-Build started
> Forkjoin validation is slow when there are many actions in chain
>
>
> Key: OOZIE-3561
> URL: https://issues.apache.org/jira/browse/OOZIE-3561
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: performance
> Attachments: OOZIE-3561-002.patch, OOZIE-3561_001.patch
>
>
> In case we have a workflow which has, let's say, 80 actions after each other:
> {{a1 -> a2 -> ... a80}}
> then the validator code "never" finishes.
> Currently the validation (in my understanding) does depth first checks from
> the start node and runs in time of n! . This is confirmed as when we split
> this huge workflow into two 40-element workflow then we get 2x ~40!-step in
> validation instead of ~80! steps.
> Guys, could you please confirm or disprove my theory?
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (OOZIE-3561) Forkjoin validation is slow when there are many actions in chain
[
https://issues.apache.org/jira/browse/OOZIE-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16981480#comment-16981480
]
Peter Bacsko commented on OOZIE-3561:
-
_"Just store the {{NodeDef}} object in the set, not a string. That should
exhibit the exact same behavior."_
After doing this locally, it turned out that this alone is not sufficient. We
have to:
# Move the memoization part a bit
# Don't store End and Join nodes
Now all existing tests pass.
> Forkjoin validation is slow when there are many actions in chain
>
>
> Key: OOZIE-3561
> URL: https://issues.apache.org/jira/browse/OOZIE-3561
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: performance
> Attachments: OOZIE-3561-002.patch, OOZIE-3561_001.patch
>
>
> In case we have a workflow which has, let's say, 80 actions after each other:
> {{a1 -> a2 -> ... a80}}
> then the validator code "never" finishes.
> Currently the validation (in my understanding) does depth first checks from
> the start node and runs in time of n! . This is confirmed as when we split
> this huge workflow into two 40-element workflow then we get 2x ~40!-step in
> validation instead of ~80! steps.
> Guys, could you please confirm or disprove my theory?
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (OOZIE-3561) Forkjoin validation is slow when there are many actions in chain
[
https://issues.apache.org/jira/browse/OOZIE-3561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated OOZIE-3561:
Attachment: OOZIE-3561-002.patch
> Forkjoin validation is slow when there are many actions in chain
>
>
> Key: OOZIE-3561
> URL: https://issues.apache.org/jira/browse/OOZIE-3561
> Project: Oozie
> Issue Type: Bug
> Components: core
>Affects Versions: 5.1.0
>Reporter: Denes Bodo
>Assignee: Denes Bodo
>Priority: Critical
> Labels: performance
> Attachments: OOZIE-3561-002.patch, OOZIE-3561_001.patch
>
>
> In case we have a workflow which has, let's say, 80 actions after each other:
> {{a1 -> a2 -> ... a80}}
> then the validator code "never" finishes.
> Currently the validation (in my understanding) does depth first checks from
> the start node and runs in time of n! . This is confirmed as when we split
> this huge workflow into two 40-element workflow then we get 2x ~40!-step in
> validation instead of ~80! steps.
> Guys, could you please confirm or disprove my theory?
> Thanks
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] Release Oozie 5.2.0 (candidate 0)
Hi all, I did the following: 1. Downloaded the package 2. Built the distro with tests (didn't see failures) under Ubuntu 18.04, JDK8 3. Integrated Oozie with 3-node Hadoop cluster running version 2.9.2 4. Run the MapReduce example I didn't have much time to do anything else. I'm giving a +1 regardless, but would like to see results from additional people to have more confidence. Regards, Peter On Mon, Nov 25, 2019 at 9:55 AM Andras Salamon wrote: > Hi All, > > We still have only one vote (+1), so I extend the voting period again. > Please cast your vote till Dec 1st. > > Best, > Sala > > On Mon, Nov 18, 2019 at 10:41 AM Andras Salamon > wrote: > > > Hi All, > > > > We only have one vote so far (+1), so I extend the voting till Nov 24th. > > Please test 5.2.0 RC0 if you have time. > > > > Best, > > Sala > > > > On Thu, Nov 14, 2019 at 3:32 PM Peter Cseh > > > wrote: > > > >> Andras, thanks for building the RC and the summary! > >> I don't think that the Java 11 issues are a problem, especially if it > >> works > >> when compiling with a different Hadoop. > >> > >> The checksums and signatures are good. > >> I've used Jdk8u212 on my mac to build the distro. > >> Distro creation succeeded with all tests passing: > >> ./bin/mkdistro.sh -Dtest.timeout=7200 -Doozie.test.waitfor.ratio=3 > >> -Dsurefire.rerunFailingTestsCount=3 > >> [INFO] > >> > >> [INFO] Reactor Summary for Apache Oozie Main 5.2.0: > >> [INFO] > >> [INFO] Apache Oozie Main .. SUCCESS [ > >> 1.538 s] > >> [INFO] Apache Oozie Fluent Job SUCCESS [ > >> 0.082 s] > >> [INFO] Apache Oozie Fluent Job API SUCCESS [ > >> 13.131 s] > >> [INFO] Apache Oozie Client SUCCESS [ > >> 10.897 s] > >> [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ > >> 6.554 s] > >> [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ > >> 3.479 s] > >> [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ > >> 0.388 s] > >> [INFO] Apache Oozie Core .. SUCCESS [ > >> 01:27 h] > >> [INFO] Apache Oozie Share Lib Streaming ... SUCCESS > [12:50 > >> min] > >> [INFO] Apache Oozie Share Lib Pig . SUCCESS > [05:38 > >> min] > >> [INFO] Apache Oozie Share Lib Git . SUCCESS [ > >> 59.980 s] > >> [INFO] Apache Oozie Share Lib Hive SUCCESS > [02:41 > >> min] > >> [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS > [02:20 > >> min] > >> [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS > [05:14 > >> min] > >> [INFO] Apache Oozie Examples .. SUCCESS [ > >> 33.237 s] > >> [INFO] Apache Oozie Share Lib Spark ... SUCCESS > [02:10 > >> min] > >> [INFO] Apache Oozie Share Lib . SUCCESS [ > >> 16.819 s] > >> [INFO] Apache Oozie Docs .. SUCCESS [ > >> 0.388 s] > >> [INFO] Apache Oozie WebApp SUCCESS [ > >> 4.631 s] > >> [INFO] Apache Oozie Tools . SUCCESS [ > >> 47.751 s] > >> [INFO] Apache Oozie MiniOozie . SUCCESS [ > >> 56.070 s] > >> [INFO] Apache Oozie Fluent Job Client . SUCCESS [ > >> 48.696 s] > >> [INFO] Apache Oozie Server SUCCESS [ > >> 5.554 s] > >> [INFO] Apache Oozie Distro SUCCESS [ > >> 39.502 s] > >> [INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [ > >> 47.454 s] > >> [INFO] > >> > >> [INFO] BUILD SUCCESS > >> [INFO] > >> > >> [INFO] Total time: 02:05 h > >> [INFO] Finished at: 2019-11-14T14:46:01+01:00 > >> [INFO] > >> > >> > >> Oozie distro created, DATE[2019.11.14-11:40:54GMT] VC-REV[unavailable], > >> available at [/Users/gezapeti/tmp/oozie-5.2.0/distro/target] > >> > >> I've also set up a local Hadoop (2.6.0) and managed to submit some > jobs. I > >> don't have the time to run a kerberized test unfortunately. > >> > >> +1 from my side. > >> > >> > >> > >> > >> On Thu, Nov 14, 2019 at 11:27 AM Andras Salamon > >> wrote: > >> > >> > Hi, > >> > > >> > I have tested the Java 10/11 compilation and found the following: > >> > > >> > Oozie cannot be compiled if we use Java 10/11 + Linux + Hadoop 2.6.x > >> > > >> > It's possible to compile using Java 11 + Mac + Hadoop 2.6.x or Java > 11 + > >> > Linux + Hadoop 2.7.x > >> > > >> > I have opened a jira about this: > >> > https://issues.apache.org/jira/browse/OOZIE-3558 > >> > > >> > Best, > >> > Sala >
Re: [VOTE] Release Oozie 5.2.0 (candidate 0)
Hi All, We still have only one vote (+1), so I extend the voting period again. Please cast your vote till Dec 1st. Best, Sala On Mon, Nov 18, 2019 at 10:41 AM Andras Salamon wrote: > Hi All, > > We only have one vote so far (+1), so I extend the voting till Nov 24th. > Please test 5.2.0 RC0 if you have time. > > Best, > Sala > > On Thu, Nov 14, 2019 at 3:32 PM Peter Cseh > wrote: > >> Andras, thanks for building the RC and the summary! >> I don't think that the Java 11 issues are a problem, especially if it >> works >> when compiling with a different Hadoop. >> >> The checksums and signatures are good. >> I've used Jdk8u212 on my mac to build the distro. >> Distro creation succeeded with all tests passing: >> ./bin/mkdistro.sh -Dtest.timeout=7200 -Doozie.test.waitfor.ratio=3 >> -Dsurefire.rerunFailingTestsCount=3 >> [INFO] >> >> [INFO] Reactor Summary for Apache Oozie Main 5.2.0: >> [INFO] >> [INFO] Apache Oozie Main .. SUCCESS [ >> 1.538 s] >> [INFO] Apache Oozie Fluent Job SUCCESS [ >> 0.082 s] >> [INFO] Apache Oozie Fluent Job API SUCCESS [ >> 13.131 s] >> [INFO] Apache Oozie Client SUCCESS [ >> 10.897 s] >> [INFO] Apache Oozie Share Lib Oozie ... SUCCESS [ >> 6.554 s] >> [INFO] Apache Oozie Share Lib HCatalog SUCCESS [ >> 3.479 s] >> [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ >> 0.388 s] >> [INFO] Apache Oozie Core .. SUCCESS [ >> 01:27 h] >> [INFO] Apache Oozie Share Lib Streaming ... SUCCESS [12:50 >> min] >> [INFO] Apache Oozie Share Lib Pig . SUCCESS [05:38 >> min] >> [INFO] Apache Oozie Share Lib Git . SUCCESS [ >> 59.980 s] >> [INFO] Apache Oozie Share Lib Hive SUCCESS [02:41 >> min] >> [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS [02:20 >> min] >> [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS [05:14 >> min] >> [INFO] Apache Oozie Examples .. SUCCESS [ >> 33.237 s] >> [INFO] Apache Oozie Share Lib Spark ... SUCCESS [02:10 >> min] >> [INFO] Apache Oozie Share Lib . SUCCESS [ >> 16.819 s] >> [INFO] Apache Oozie Docs .. SUCCESS [ >> 0.388 s] >> [INFO] Apache Oozie WebApp SUCCESS [ >> 4.631 s] >> [INFO] Apache Oozie Tools . SUCCESS [ >> 47.751 s] >> [INFO] Apache Oozie MiniOozie . SUCCESS [ >> 56.070 s] >> [INFO] Apache Oozie Fluent Job Client . SUCCESS [ >> 48.696 s] >> [INFO] Apache Oozie Server SUCCESS [ >> 5.554 s] >> [INFO] Apache Oozie Distro SUCCESS [ >> 39.502 s] >> [INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [ >> 47.454 s] >> [INFO] >> >> [INFO] BUILD SUCCESS >> [INFO] >> >> [INFO] Total time: 02:05 h >> [INFO] Finished at: 2019-11-14T14:46:01+01:00 >> [INFO] >> >> >> Oozie distro created, DATE[2019.11.14-11:40:54GMT] VC-REV[unavailable], >> available at [/Users/gezapeti/tmp/oozie-5.2.0/distro/target] >> >> I've also set up a local Hadoop (2.6.0) and managed to submit some jobs. I >> don't have the time to run a kerberized test unfortunately. >> >> +1 from my side. >> >> >> >> >> On Thu, Nov 14, 2019 at 11:27 AM Andras Salamon >> wrote: >> >> > Hi, >> > >> > I have tested the Java 10/11 compilation and found the following: >> > >> > Oozie cannot be compiled if we use Java 10/11 + Linux + Hadoop 2.6.x >> > >> > It's possible to compile using Java 11 + Mac + Hadoop 2.6.x or Java 11 + >> > Linux + Hadoop 2.7.x >> > >> > I have opened a jira about this: >> > https://issues.apache.org/jira/browse/OOZIE-3558 >> > >> > Best, >> > Sala >> > >> > On Mon, Nov 11, 2019 at 8:44 PM Attila Sasvári >> > wrote: >> > >> > > I failed to build Oozie 5.2.0 rc0 with OpenJDK 10 and OpenJDK 8. >> > > >> > > # Oozie 5.2.0 rc0 with Java 10 : >> > > - downloaded >> > > >> https://people.apache.org/~asalamon74/oozie-5.2.0-rc0/oozie-5.2.0.tar.gz >> > > - extracted it to oozie-5.2.0 >> > > - based on the public OpenJDK 10 Docker image ( >> > > https://hub.docker.com/_/openjdk), created the following Dockerfile >> to >> > > make >> > > the Oozie distro: >> > > >> > > FROM openjdk:10 >> > > RUN apt update -y >> > > RUN apt install -y maven >> > > COPY oozie-5.2.0 /usr/src/oozie >> > > WORKDIR /usr/src/oozie >> > > RUN bin/mkdistro.sh -DskipTests >> > > >> > > - doc
