Re: [VOTE] Release Oozie 5.2.0 (candidate 0)
Hi all, My vote: 0 I don't want to block the release. There was one small thing I noticed: - Spark 1 example with default settings (local mode) fails $ bin/oozie job -oozie http://localhost:11000/oozie -config examples/apps/spark/job.properties -run -DnameNode=hdfs://localhost:9000 -DjobTracker=localhost:8032 Error: --deploy-mode must be either "client" or "cluster" Other things I tested: - MR & Demo example workflows succeeded - Spark 1 example workflow in yarn-cluster mode succeeded - Created a simple workflow with the shell action via Fluent Job API - succeeded . Previous issue: solved (after you run package distro, you should remove target of the fluent-job module before running mvn install) - Performed some basic UI tests Regards, - Attila ps: I didn't test Spark 2, coordinators, bundles and more advanced features. I could not compile Oozie with Java 11 target & source version using OpenJDK due to JEP-220 (removed tools.jar) & HADOOP-15338. On Sun, Dec 1, 2019 at 10:20 AM Attila Sasvári wrote: > Hi all, > > So far I tested RC 0 with openjdk 8 & a pseudo distributed Hadoop (2.9.0), > demo map reduce action succeeded, UI & logs looked OK. > > How did you test the Oozie Fluent job API 5.2.0? I know it works with Java > 8 (tools.jar is included). > > I wanted to test it, downloaded RC and tried to locally install maven > dependency, and build failed: > > mvn install -DskipTests -Dhadoop.version=2.9.0 -Puber > > [ERROR] Failed to execute goal > org.codehaus.mojo:xml-maven-plugin:1.0.1:transform (default) on project > oozie-fluent-job-api: Failed to transform input file /home > /asasvari/oozie/oozie-5.2.0/fluent-job/fluent-job-api/target/spotbugs/spotbugs-filter.html: > javax.xml.transform.TransformerException: com.sun.org.apache.xml.inte > rnal.utils.WrappedRuntimeException: The entity name must immediately > follow the '&' in the entity reference. -> [Help 1] > > Can someone else try to repro this please before moving forward? > > Thanks > - Attila > > ~/oozie/oozie-5.2.0$ java -version > openjdk version "1.8.0_222" > OpenJDK Runtime Environment (build > 1.8.0_222-8u222-b10-1ubuntu1~18.04.1-b10) > OpenJDK 64-Bit Server VM (build 25.222-b10, mixed mode) > > > On Thu, Nov 28, 2019 at 2:13 PM Julia Kinga Marton > wrote: > > > > Hi all, > > > > +1, from my side. > > > > I did the following steps: > > - checked sha > > - checked gpg > > - checked rat report > > - built Oozie with Hadoop 2.6.0 using mkdistro > > - tested some basic cli commands and run the examples in pseudo > distributed > > mode. Found two small issue related the Hive example (OOZIE-3563 > > <https://issues.apache.org/jira/browse/OOZIE-3563> > > - OOZIE-3562 <https://issues.apache.org/jira/browse/OOZIE-3562>), but > since > > we are using a very old version of Hive, I don't consider them blockers. > > > > Regards, > > Kinga > > > > On Mon, Nov 25, 2019 at 10:55 AM Peter Bacsko > > > wrote: > > > > > Hi all, > > > > > > I did the following: > > > 1. Downloaded the package > > > 2. Built the distro with tests (didn't see failures) under Ubuntu > 18.04, > > > JDK8 > > > 3. Integrated Oozie with 3-node Hadoop cluster running version 2.9.2 > > > 4. Run the MapReduce example > > > > > > I didn't have much time to do anything else. > > > > > > I'm giving a +1 regardless, but would like to see results from > additional > > > people to have more confidence. > > > > > > Regards, > > > Peter > > > > > > On Mon, Nov 25, 2019 at 9:55 AM Andras Salamon > > > wrote: > > > > > > > Hi All, > > > > > > > > We still have only one vote (+1), so I extend the voting period > again. > > > > Please cast your vote till Dec 1st. > > > > > > > > Best, > > > > Sala > > > > > > > > On Mon, Nov 18, 2019 at 10:41 AM Andras Salamon < > asala...@cloudera.com> > > > > wrote: > > > > > > > > > Hi All, > > > > > > > > > > We only have one vote so far (+1), so I extend the voting till Nov > > > 24th. > > > > > Please test 5.2.0 RC0 if you have time. > > > > > > > > > > Best, > > > > > Sala > > > > > > > > > > On Thu, Nov 14, 2019 at 3:32 PM Peter Cseh > > > > > > > > > > > > wrote: > > > > > > > > > &g
Re: [VOTE] Release Oozie 5.2.0 (candidate 0)
>> 3.479 s] > > > >> [INFO] Apache Oozie Share Lib Distcp .. SUCCESS [ > > > >> 0.388 s] > > > >> [INFO] Apache Oozie Core .. SUCCESS [ > > > >> 01:27 h] > > > >> [INFO] Apache Oozie Share Lib Streaming ... SUCCESS > > > [12:50 > > > >> min] > > > >> [INFO] Apache Oozie Share Lib Pig . SUCCESS > > > [05:38 > > > >> min] > > > >> [INFO] Apache Oozie Share Lib Git . SUCCESS [ > > > >> 59.980 s] > > > >> [INFO] Apache Oozie Share Lib Hive SUCCESS > > > [02:41 > > > >> min] > > > >> [INFO] Apache Oozie Share Lib Hive 2 .. SUCCESS > > > [02:20 > > > >> min] > > > >> [INFO] Apache Oozie Share Lib Sqoop ... SUCCESS > > > [05:14 > > > >> min] > > > >> [INFO] Apache Oozie Examples .. SUCCESS [ > > > >> 33.237 s] > > > >> [INFO] Apache Oozie Share Lib Spark ... SUCCESS > > > [02:10 > > > >> min] > > > >> [INFO] Apache Oozie Share Lib . SUCCESS [ > > > >> 16.819 s] > > > >> [INFO] Apache Oozie Docs .. SUCCESS [ > > > >> 0.388 s] > > > >> [INFO] Apache Oozie WebApp SUCCESS [ > > > >> 4.631 s] > > > >> [INFO] Apache Oozie Tools . SUCCESS [ > > > >> 47.751 s] > > > >> [INFO] Apache Oozie MiniOozie . SUCCESS [ > > > >> 56.070 s] > > > >> [INFO] Apache Oozie Fluent Job Client . SUCCESS [ > > > >> 48.696 s] > > > >> [INFO] Apache Oozie Server SUCCESS [ > > > >> 5.554 s] > > > >> [INFO] Apache Oozie Distro SUCCESS [ > > > >> 39.502 s] > > > >> [INFO] Apache Oozie ZooKeeper Security Tests .. SUCCESS [ > > > >> 47.454 s] > > > >> [INFO] > > > >> > > > > > >> [INFO] BUILD SUCCESS > > > >> [INFO] > > > >> > > > > > >> [INFO] Total time: 02:05 h > > > >> [INFO] Finished at: 2019-11-14T14:46:01+01:00 > > > >> [INFO] > > > >> > > > > > >> > > > >> Oozie distro created, DATE[2019.11.14-11:40:54GMT] > > VC-REV[unavailable], > > > >> available at [/Users/gezapeti/tmp/oozie-5.2.0/distro/target] > > > >> > > > >> I've also set up a local Hadoop (2.6.0) and managed to submit some > > > jobs. I > > > >> don't have the time to run a kerberized test unfortunately. > > > >> > > > >> +1 from my side. > > > >> > > > >> > > > >> > > > >> > > > >> On Thu, Nov 14, 2019 at 11:27 AM Andras Salamon > > > >> wrote: > > > >> > > > >> > Hi, > > > >> > > > > >> > I have tested the Java 10/11 compilation and found the following: > > > >> > > > > >> > Oozie cannot be compiled if we use Java 10/11 + Linux + Hadoop 2.6.x > > > >> > > > > >> > It's possible to compile using Java 11 + Mac + Hadoop 2.6.x or Java > > > 11 + > > > >> > Linux + Hadoop 2.7.x > > > >> > > > > >> > I have opened a jira about this: > > > >> > https://issues.apache.org/jira/browse/OOZIE-3558 > > > >> > > > > >> > Best, > > > >> > Sala > > > >> > > > > >> > On Mon, Nov 11, 2019 at 8:44 PM Attila Sasvári < asasv...@apache.org > > > > > > >> > wrote: > > > >> > > > > >> > > I failed to build Oozie 5.2.0 rc0 with OpenJDK 10 and OpenJDK 8. > > > >> > > > > > >> > > # Oozie 5.2.0 rc0
Re: [VOTE] Release Oozie 5.2.0 (candidate 0)
Update: using maven 3.6.2, I managed to build the distro with OpenJDK 8. Apache Maven 3.3.9 might be too old (we should document minimum maven version). On Mon, Nov 11, 2019 at 7:44 PM Attila Sasvári wrote: > I failed to build Oozie 5.2.0 rc0 with OpenJDK 10 and OpenJDK 8. > > # Oozie 5.2.0 rc0 with Java 10 : > - downloaded > https://people.apache.org/~asalamon74/oozie-5.2.0-rc0/oozie-5.2.0.tar.gz > - extracted it to oozie-5.2.0 > - based on the public OpenJDK 10 Docker image ( > https://hub.docker.com/_/openjdk), created the following Dockerfile to > make the Oozie distro: > > FROM openjdk:10 > RUN apt update -y > RUN apt install -y maven > COPY oozie-5.2.0 /usr/src/oozie > WORKDIR /usr/src/oozie > RUN bin/mkdistro.sh -DskipTests > > - docker build -f Dockerfile . failed with > > [ERROR] Failed to execute goal on project oozie-fluent-job-api: Could not > resolve dependencies for project > org.apache.oozie:oozie-fluent-job-api:jar:5.2.0: Could not find artifact > jdk.tools:jdk.tools:jar:1.6 at specified path > /usr/lib/jvm/java-10-openjdk-amd64/../lib/tools.jar -> [Help 1] > > when it tried to build the Fluent Job API. > > - Fluent Job API's jaxb dependency seems to rely on the existence of > tools.jar. However, JEP-220 (https://openjdk.java.net/jeps/220) removed > tools.jar and it affects JDK 9+ environments. > > # Oozie 5.2.0 rc0 with Java 8, using the OpenJDK 8 base Docker image: > - Dockerfile > FROM openjdk:8 > RUN apt update -y > RUN apt install -y maven > COPY oozie-5.2.0 /usr/src/oozie > WORKDIR /usr/src/oozie > RUN bin/mkdistro.sh -DskipTests > > Building Apache Oozie WebApp failed with a different error: > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-dependency-plugin:3.0.2:copy (copy-sharelib) > on project oozie-webapp: Unable to find/resolve artifact. > com.google.inject.ProvisionException: Unable to provision, see the > following errors: > [ERROR] > [ERROR] 1) Error injecting: private org.eclipse.aether.spi.log.Logger > org.apache.maven.repository.internal.DefaultVersionRangeResolver.logger > [ERROR] while locating > org.apache.maven.repository.internal.DefaultVersionRangeResolver > > Has anyone else experienced similar build failures? > > Regards, > - Attila > > On Sun, Nov 10, 2019 at 10:59 PM Attila Sasvári > wrote: > > > > Hi Sala, > > > > Many thanks for driving the release. > > > > Can you please outline main changes and new features of Oozie 5.2? > > > > Best, > > - Attila > > > > On Fri, Nov 8, 2019 at 11:15 AM Andras Salamon > wrote: > >> > >> Hi, > >> > >> I have created a build for Oozie 5.2.0, candidate 0. > >> > >> Keys to verify the signature of the release artifact are available at > >> > >> https://www.apache.org/dist/oozie/KEYS > >> > >> Please download, test, and try it out: > >> > >> https://people.apache.org/~asalamon74/oozie-5.2.0-rc0/ > >> > >> The release, sha512 signature, gpg signature, and rat report can all > >> be found at the above address. > >> > >> Vote closes on Nov 17th. > >> > >> Thanks, > >> Sala >
Re: [VOTE] Release Oozie 5.2.0 (candidate 0)
I failed to build Oozie 5.2.0 rc0 with OpenJDK 10 and OpenJDK 8. # Oozie 5.2.0 rc0 with Java 10 : - downloaded https://people.apache.org/~asalamon74/oozie-5.2.0-rc0/oozie-5.2.0.tar.gz - extracted it to oozie-5.2.0 - based on the public OpenJDK 10 Docker image ( https://hub.docker.com/_/openjdk), created the following Dockerfile to make the Oozie distro: FROM openjdk:10 RUN apt update -y RUN apt install -y maven COPY oozie-5.2.0 /usr/src/oozie WORKDIR /usr/src/oozie RUN bin/mkdistro.sh -DskipTests - docker build -f Dockerfile . failed with [ERROR] Failed to execute goal on project oozie-fluent-job-api: Could not resolve dependencies for project org.apache.oozie:oozie-fluent-job-api:jar:5.2.0: Could not find artifact jdk.tools:jdk.tools:jar:1.6 at specified path /usr/lib/jvm/java-10-openjdk-amd64/../lib/tools.jar -> [Help 1] when it tried to build the Fluent Job API. - Fluent Job API's jaxb dependency seems to rely on the existence of tools.jar. However, JEP-220 (https://openjdk.java.net/jeps/220) removed tools.jar and it affects JDK 9+ environments. # Oozie 5.2.0 rc0 with Java 8, using the OpenJDK 8 base Docker image: - Dockerfile FROM openjdk:8 RUN apt update -y RUN apt install -y maven COPY oozie-5.2.0 /usr/src/oozie WORKDIR /usr/src/oozie RUN bin/mkdistro.sh -DskipTests Building Apache Oozie WebApp failed with a different error: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-dependency-plugin:3.0.2:copy (copy-sharelib) on project oozie-webapp: Unable to find/resolve artifact. com.google.inject.ProvisionException: Unable to provision, see the following errors: [ERROR] [ERROR] 1) Error injecting: private org.eclipse.aether.spi.log.Logger org.apache.maven.repository.internal.DefaultVersionRangeResolver.logger [ERROR] while locating org.apache.maven.repository.internal.DefaultVersionRangeResolver Has anyone else experienced similar build failures? Regards, - Attila On Sun, Nov 10, 2019 at 10:59 PM Attila Sasvári wrote: > > Hi Sala, > > Many thanks for driving the release. > > Can you please outline main changes and new features of Oozie 5.2? > > Best, > - Attila > > On Fri, Nov 8, 2019 at 11:15 AM Andras Salamon wrote: >> >> Hi, >> >> I have created a build for Oozie 5.2.0, candidate 0. >> >> Keys to verify the signature of the release artifact are available at >> >> https://www.apache.org/dist/oozie/KEYS >> >> Please download, test, and try it out: >> >> https://people.apache.org/~asalamon74/oozie-5.2.0-rc0/ >> >> The release, sha512 signature, gpg signature, and rat report can all >> be found at the above address. >> >> Vote closes on Nov 17th. >> >> Thanks, >> Sala
Re: [VOTE] Release Oozie 5.2.0 (candidate 0)
Hi Sala, Many thanks for driving the release. Can you please outline main changes and new features of Oozie 5.2? Best, - Attila On Fri, Nov 8, 2019 at 11:15 AM Andras Salamon wrote: > Hi, > > I have created a build for Oozie 5.2.0, candidate 0. > > Keys to verify the signature of the release artifact are available at > > https://www.apache.org/dist/oozie/KEYS > > Please download, test, and try it out: > > https://people.apache.org/~asalamon74/oozie-5.2.0-rc0/ > > The release, sha512 signature, gpg signature, and rat report can all > be found at the above address. > > Vote closes on Nov 17th. > > Thanks, > Sala >
[ANNOUNCE] Apache Oozie 5.0.0-beta1 released
Hello Oozie Community, The Apache Oozie team is pleased to announce the release of Oozie 5.0.0-beta1. Oozie 5.0.0-beta1 includes new features, bug fixes and minor improvements. The details of the release and download location can be found at http://oozie.apache.org/ The full release notes can be found at the download location, but some of the highlights of the release include: - Migrated Oozie Launcher from MapReduce to YARN AM for significant performance improvements and simplicity - Dropped support for Hadoop 1.x - Added new 1.0 workflow schemas - Switched from Tomcat to embedded Jetty for easier management and better security - Completely rewritten Job DAG renderer to support SVG and DOT formats as well as better performance - Better resiliency to transient database outages - Requires Java 8 Many thanks to all the developers, QA's, users and committers for making this release happen. Regards, - Attila
[VOTE] Release Oozie 5.0.0-beta1 (candidate 2)
Hi, I have created a build for Oozie 5.0.0-beta1, release candidate 2. This release introduces major changes: - moving launcher from MapReduce AM to Oozie AM, - dropped support for Hadoop 1.x, - added new 1.0 workflow schemas, - switching from Tomcat to embedded Jetty, - completely rewritten graph generator, - local sharelib, - JDK 8 support. It also includes OOZIE-3083, OOZIE-3159 (beta1 blockers). Keys to verify the signature of the release artifact are available at http://www.apache.org/dist/oozie/KEYS Please download, test, and try it out: http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc2/ The release, md5 signature, gpg signature, and rat report can all be found at the above address. The vote is open until Tuesday, January 23, 2018 at 9:00 CEST. Thanks, Attila Sasvari
Re: [VOTE] Release Oozie 5.0.0-beta1 (candidate 1)
It is release candidate 1. On Tue, Jan 9, 2018 at 5:50 PM, Attila Sasvári <at.sasv...@gmail.com> wrote: > Hi, > > I have created a build for Oozie 5.0.0-beta1, release candidate 0. > > This release introduces major changes: > - moving launcher from MapReduce AM to Oozie AM, > - dropped support for Hadoop 1.x, > - added new 1.0 workflow schemas, > - switching from Tomcat to embedded Jetty, > - completely rewritten graph generator, > - local sharelib, > - JDK 8 support. > > It also includes OOZIE-3083. > > Keys to verify the signature of the release artifact are available at > > http://www.apache.org/dist/oozie/KEYS > > Please download, test, and try it out: > > http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc1/ > > The release, md5 signature, gpg signature, and rat report can all be found > at the above address. > > The vote is open until Friday, January 12, 2018 at 18:00 CEST. > > Thanks, > Attila Sasvari >
[VOTE] Release Oozie 5.0.0-beta1 (candidate 1)
Hi, I have created a build for Oozie 5.0.0-beta1, release candidate 0. This release introduces major changes: - moving launcher from MapReduce AM to Oozie AM, - dropped support for Hadoop 1.x, - added new 1.0 workflow schemas, - switching from Tomcat to embedded Jetty, - completely rewritten graph generator, - local sharelib, - JDK 8 support. It also includes OOZIE-3083. Keys to verify the signature of the release artifact are available at http://www.apache.org/dist/oozie/KEYS Please download, test, and try it out: http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc1/ The release, md5 signature, gpg signature, and rat report can all be found at the above address. The vote is open until Friday, January 12, 2018 at 18:00 CEST. Thanks, Attila Sasvari
[VOTE] Release Oozie 5.0.0-beta1 (candidate 0)
Hi there, I have created a build for Oozie 5.0.0-beta1, release candidate 0. This release introduces major changes: - moving launcher from MapReduce AM to Oozie AM, - switching from Tomcat to embedded Jetty, - completely rewritten graph generator, - local sharelib, - JDK 8 support. Keys to verify the signature of the release artifact are available at http://www.apache.org/dist/oozie/KEYS Please download, test, and try it out: http://people.apache.org/~asasvari/oozie-5.0.0-beta1-rc0/ The release, md5 signature, gpg signature, and rat report can all be found at the above address. The vote is open until Thursday, January 04, 2018 at 12:00 CEST. Thanks, Attila Sasvari
