Hi All, Below is the list of issues, picked from currently available patches. I could not decide up any 'open' issue, If there is anything, please reply here.
We can 'drop/add' more if required based on the feedback. Important is, how many of these we will be able to review and commit. I will starting looking into these issues and provide feedback. --------------------------------------------------------------------------- OOZIE-1978 Forkjoin validation code is ridiculously slow in some cases OOZIE-2508 Documentation change for Coord action rerun [OOZIE-1735] OOZIE-2583 oozie throws EL Exception when reference variable name containing dot OOZIE-2582 Populating external child Ids for action failures OOZIE-2517 Add support for startCreatedTime and endCreatedTime filters for coord and bundles OOZIE-2512 ShareLibservice returns incorrect path for jar OOZIE-2572 SLA DURATION miss not shown when job is running for longer than expected time OOZIE-2457 Oozie log parsing regex consume more than 90% cpu OOZIE-2574 Oozie to support replication-enabled mysql urls OOZIE-2506 Add logs into RecoverService for logging information about queued commnads OOZIE-2573 dataset url which contains spaces can not be handled rightly OOZIE-2569 Adding yarn-site, core-site, hdfs-site and mapred-site into spark launcher OOZIE-2491 oozie acl cannot specify group,it does`t work OOZIE-2539 Incorrect property key is used for 'hive log4j configuration file for execution mode' OOZIE-2555 Oozie SSL enable setup does not return port for admin -servers OOZIE-2515 Duplicate information for "Changing endtime/pausetime of a Bundle Job" in CommandLineTool wiki OOZIE-2498 Oozie CallerId configuration for downstream components OOZIE-2516 Update web service documentation for jobs API OOZIE-2500 -DtestJarSimple option mentioned in minioozie doc does not work OOZIE-2513 log.scan.duration should not be used for error and audit logs OOZIE-2461 Workflow, Coordinator and Bundle job querying should have last modified filter OOZIE-2306 Oozie health check OOZIE-2518 Oozie is creating many files and directories in /tmp per day OOZIE-2409 hive2 action with hive 1.2.1 failed OOZIE-2473 Connection pool for SMTP connection OOZIE-2440 Exponential re-try policy for workflow action OOZIE-2458 'oozie-setup.sh sharelib create' should ensure uploaded jars are world readable OOZIE-2390 Rerun with failed option removing completed output data OOZIE-1810 Workflow cannot get into Failed state when kill control node cannot resolve variable in message OOZIE-2244 Oozie should mask passwords in the logs when logging command arguments --------------------------------------------------------------------------- Thanks, Abhishek > On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <[email protected]> wrote: > > Hi All, > > I looked into the commits (approx. 160), all looked good to me and should be > included into the release. If there are things, which needs to be excluded, > provide your comments. > > We still need to decide on the patch which are in progress status. > > On the upgrade side: > Java version 1.6 to 1.7 > OpenJPA version 2.2.2 to 2.4.1 > Curator version 2.5.0 to 2.11.0 > Pig version 0.12.1 to 0.16.0 > Hive version 0.13.1 to 1.2.0 > HttpClient version 4.3 to 4.3.6 > commons-httpclient version 3.1 to 4.3.6 (discontinued) > > Please provide your feedback on the above upgrades. > > Once we decide on the scope of the release. We will start working on the > completing/testing the release. > > Thanks, > Abhishek > >> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <[email protected] >> <mailto:[email protected]>> wrote: >> >> Okay. We will make the Java version to 1.7. >> >> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc. >> >> HttpClient, Pig (0.16) version upgrade we have done internally as well. >> >> As per the comments on the OOZIE-2231 >> <https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also can >> be upgraded to 2.11. >> >> Thanks, >> Abhishek >>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> We've been using a version of Hive/HCat based on 1.1.0 for over a year now >>> with no problems. We did have to comment out a few lines in MiniHCatServer >>> and MiniHS2 that were no longer needed: >>> >>> diff --git a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>> index 8699ff8..d694977 100644 >>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>> @@ -150,9 +150,10 @@ public class MiniHCatServer { >>> public static void resetDefaultDBCreation() throws Exception { >>> // Need to do this, else default db will not be created for local >>> metastores. >>> // TestHiveMain will fail with >>> InvalidObjectException(message:There is no database named default) >>> - Field declaredField = >>> HMSHandler.class.getDeclaredField("createDefaultDB"); >>> - declaredField.setAccessible(true); >>> - declaredField.set(null, false); >>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later >>> + //Field declaredField = >>> HMSHandler.class.getDeclaredField("createDefaultDB"); >>> + //declaredField.setAccessible(true); >>> + //declaredField.set(null, false); >>> } >>> >>> public static void resetHiveConfStaticVariables() throws Exception { >>> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>> index c9dfd56..505dc55 100644 >>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService { >>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost()); >>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, getBinaryPort()); >>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT, >>> getHttpPort()); >>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag(); >>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later >>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag(); >>> >>> Path scratchDir = new Path(baseDfsDir, "scratch"); >>> fs.mkdirs(scratchDir); >>> >>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan < >>> [email protected] <mailto:[email protected]>> wrote: >>> >>>> Let us explore changing the default hive version also to 1.x. 0.13 is >>>> old >>>> >>>> Thanks >>>> >>>> Venkat >>>> >>>> >>>> On 7/15/16, 11:31 AM, "Robert Kanter" <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> I don't think we have any guidelines defined anywhere about when it's >>>> okay >>>> to change the minimum Java version, but perhaps we need to wait until >>>> the >>>> next major release (5.0)? That said, Hadoop did this in a minor >>>> release >>>> and Java 6 was EoL in 2013. So I'm fine with doing it in 4.3 if >>>> nobody has >>>> an objections; we should just make sure to call that out in our release >>>> notes. >>>> >>>> Otherwise, nothing comes to mind for anything we should leave out other >>>> than the Oozie on Yarn stuff, which is not currently in trunk anyway. >>>> >>>> I know there's a number of Spark Action JIRAs; we should make sure to >>>> include all of these and any pending ones. >>>> @Rohini, @Puru, @Satish, anything you think needs more time to bake >>>> that we >>>> should hold off on including in the release? >>>> >>>> >>>> thanks >>>> - Robert >>>> >>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna <[email protected] >>>> <mailto:[email protected]>> >>>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> Any suggestions or comments on the scope for the release? >>>>> >>>>> Thanks, >>>>> Abhishek >>>>> >>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <[email protected] >>>>>> <mailto:[email protected]>> >>>>> wrote: >>>>>> >>>>>> Hi All, >>>>>> >>>>>> Oozie is starting the release process for Oozie 4.3.0. This thread >>>> is >>>>> for discussion about the scope of the release. Please provide your >>>>> suggestion about what should be included (or any exclusions) in the >>>> release. >>>>>> >>>>>> Currently, we have around 158 patches submitted to trunk. Based on >>>> the >>>>> scope, we will include (exclude) more if required. >>>>>> Version Upgrade >>>>>> Java 1.7 (OOZIE-2036 < >>>> https://issues.apache.org/jira/browse/OOZIE-2036 >>>> <https://issues.apache.org/jira/browse/OOZIE-2036>>) >>>>>> OpenJPA (OOZIE-2488 < >>>> https://issues.apache.org/jira/browse/OOZIE-2488 >>>> <https://issues.apache.org/jira/browse/OOZIE-2488>>), >>>>>> Curator (OOZIE-2231 < >>>> https://issues.apache.org/jira/browse/OOZIE-2231 >>>> <https://issues.apache.org/jira/browse/OOZIE-2231> >>>>>> ), >>>>>> HttpClient (OOZIE-2538 < >>>>> https://issues.apache.org/jira/browse/OOZIE-2538 >>>>> <https://issues.apache.org/jira/browse/OOZIE-2538>>), >>>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602 >>>>>> <https://issues.apache.org/jira/browse/OOZIE-2602> >>>>> ). >>>>>> New Features/Improvements: Add default JT and NN, Database schema >>>>> service check, instrumental metrics, Tomcat upgrade etc. >>>>>> Bug Fixes. >>>>>> Provide your opinion on inclusion/exclusion of the above points or >>>> more >>>>> for the scope of the release. We can also, take some work which is >>>> already >>>>> done and review is pending for them (depending on the availability >>>> of the >>>>> reviewers). >>>>>> >>>>>> Thanks, >>>>>> Abhishek >>>>> >>>>> >>>> >>>> >>>> >> >
