That list looks good to me. The only one I'm not sure of is OOZIE-2306 (Oozie Health Check). It's a new feature and it might make sense to let it "bake" a while first? I'm also not sure it's quite ready yet.
I'd also like to add a few others to the list: - OOZIE-2273 (MiniOozie does not work outside of Oozie) - The subtasks under OOZIE-2036 (Drop support for Java 1.6) --- OOZIE-2037 (Add TLSv1.1,TLSv1.2) --- OOZIE-1793 (Improve find bugs reporting for Oozie) --- OOZIE-2552 (Update ActiveMQ version for security and other fixes) There's a number of other Patch Available JIRAs not on your list (including the ones I mentioned above). While we don't need to include everything here (and we probably shouldn't nor do we have the review bandwidth), did you consider each of them when creating the list? I currently see 97 JIRAs: https://s.apache.org/oozie-patch-available I think we should include all Spark Action fixes/improvements, given how popular it seems to be. thanks - Robert On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <[email protected]> wrote: > Hi All, > > Below is the list of issues, picked from currently available patches. I > could not decide up any 'open' issue, If there is anything, please reply > here. > > We can 'drop/add' more if required based on the feedback. Important is, > how many of these we will be able to review and commit. > > I will starting looking into these issues and provide feedback. > --------------------------------------------------------------------------- > OOZIE-1978 Forkjoin validation code is ridiculously slow in some cases > OOZIE-2508 Documentation change for Coord action rerun [OOZIE-1735] > OOZIE-2583 oozie throws EL Exception when reference variable name > containing dot > OOZIE-2582 Populating external child Ids for action failures > OOZIE-2517 Add support for startCreatedTime and endCreatedTime > filters for coord and bundles > OOZIE-2512 ShareLibservice returns incorrect path for jar > OOZIE-2572 SLA DURATION miss not shown when job is running for longer > than expected time > OOZIE-2457 Oozie log parsing regex consume more than 90% cpu > OOZIE-2574 Oozie to support replication-enabled mysql urls > OOZIE-2506 Add logs into RecoverService for logging information about > queued commnads > OOZIE-2573 dataset url which contains spaces can not be handled > rightly > OOZIE-2569 Adding yarn-site, core-site, hdfs-site and mapred-site > into spark launcher > OOZIE-2491 oozie acl cannot specify group,it does`t work > OOZIE-2539 Incorrect property key is used for 'hive log4j > configuration file for execution mode' > OOZIE-2555 Oozie SSL enable setup does not return port for admin > -servers > OOZIE-2515 Duplicate information for "Changing endtime/pausetime of a > Bundle Job" in CommandLineTool wiki > OOZIE-2498 Oozie CallerId configuration for downstream components > OOZIE-2516 Update web service documentation for jobs API > OOZIE-2500 -DtestJarSimple option mentioned in minioozie doc does not > work > OOZIE-2513 log.scan.duration should not be used for error and audit > logs > OOZIE-2461 Workflow, Coordinator and Bundle job querying should have > last modified filter > OOZIE-2306 Oozie health check > OOZIE-2518 Oozie is creating many files and directories in /tmp per > day > OOZIE-2409 hive2 action with hive 1.2.1 failed > OOZIE-2473 Connection pool for SMTP connection > OOZIE-2440 Exponential re-try policy for workflow action > OOZIE-2458 'oozie-setup.sh sharelib create' should ensure uploaded > jars are world readable > OOZIE-2390 Rerun with failed option removing completed output data > OOZIE-1810 Workflow cannot get into Failed state when kill control > node cannot resolve variable in message > OOZIE-2244 Oozie should mask passwords in the logs when logging > command arguments > --------------------------------------------------------------------------- > Thanks, > Abhishek > > > On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <[email protected]> > wrote: > > > > Hi All, > > > > I looked into the commits (approx. 160), all looked good to me and > should be included into the release. If there are things, which needs to be > excluded, provide your comments. > > > > We still need to decide on the patch which are in progress status. > > > > On the upgrade side: > > Java version 1.6 to 1.7 > > OpenJPA version 2.2.2 to 2.4.1 > > Curator version 2.5.0 to 2.11.0 > > Pig version 0.12.1 to 0.16.0 > > Hive version 0.13.1 to 1.2.0 > > HttpClient version 4.3 to 4.3.6 > > commons-httpclient version 3.1 to 4.3.6 (discontinued) > > > > Please provide your feedback on the above upgrades. > > > > Once we decide on the scope of the release. We will start working on the > completing/testing the release. > > > > Thanks, > > Abhishek > > > >> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <[email protected] > <mailto:[email protected]>> wrote: > >> > >> Okay. We will make the Java version to 1.7. > >> > >> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig etc. > >> > >> HttpClient, Pig (0.16) version upgrade we have done internally as well. > >> > >> As per the comments on the OOZIE-2231 < > https://issues.apache.org/jira/browse/OOZIE-2231>, curator version also > can be upgraded to 2.11. > >> > >> Thanks, > >> Abhishek > >>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <[email protected] > <mailto:[email protected]>> wrote: > >>> > >>> We've been using a version of Hive/HCat based on 1.1.0 for over a year > now > >>> with no problems. We did have to comment out a few lines in > MiniHCatServer > >>> and MiniHS2 that were no longer needed: > >>> > >>> diff --git > a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>> index 8699ff8..d694977 100644 > >>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >>> @@ -150,9 +150,10 @@ public class MiniHCatServer { > >>> public static void resetDefaultDBCreation() throws Exception { > >>> // Need to do this, else default db will not be created for > local > >>> metastores. > >>> // TestHiveMain will fail with > >>> InvalidObjectException(message:There is no database named default) > >>> - Field declaredField = > >>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >>> - declaredField.setAccessible(true); > >>> - declaredField.set(null, false); > >>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later > >>> + //Field declaredField = > >>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >>> + //declaredField.setAccessible(true); > >>> + //declaredField.set(null, false); > >>> } > >>> > >>> public static void resetHiveConfStaticVariables() throws Exception > { > >>> diff --git a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>> index c9dfd56..505dc55 100644 > >>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService { > >>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, getHost()); > >>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, > getBinaryPort()); > >>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT, > >>> getHttpPort()); > >>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later > >>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >>> > >>> Path scratchDir = new Path(baseDfsDir, "scratch"); > >>> fs.mkdirs(scratchDir); > >>> > >>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan < > >>> [email protected] <mailto:[email protected]>> > wrote: > >>> > >>>> Let us explore changing the default hive version also to 1.x. 0.13 > is > >>>> old > >>>> > >>>> Thanks > >>>> > >>>> Venkat > >>>> > >>>> > >>>> On 7/15/16, 11:31 AM, "Robert Kanter" <[email protected] <mailto: > [email protected]>> wrote: > >>>> > >>>> I don't think we have any guidelines defined anywhere about when > it's > >>>> okay > >>>> to change the minimum Java version, but perhaps we need to wait > until > >>>> the > >>>> next major release (5.0)? That said, Hadoop did this in a minor > >>>> release > >>>> and Java 6 was EoL in 2013. So I'm fine with doing it in 4.3 if > >>>> nobody has > >>>> an objections; we should just make sure to call that out in our > release > >>>> notes. > >>>> > >>>> Otherwise, nothing comes to mind for anything we should leave out > other > >>>> than the Oozie on Yarn stuff, which is not currently in trunk > anyway. > >>>> > >>>> I know there's a number of Spark Action JIRAs; we should make sure > to > >>>> include all of these and any pending ones. > >>>> @Rohini, @Puru, @Satish, anything you think needs more time to bake > >>>> that we > >>>> should hold off on including in the release? > >>>> > >>>> > >>>> thanks > >>>> - Robert > >>>> > >>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna < > [email protected] <mailto:[email protected]>> > >>>> wrote: > >>>> > >>>>> Hi All, > >>>>> > >>>>> Any suggestions or comments on the scope for the release? > >>>>> > >>>>> Thanks, > >>>>> Abhishek > >>>>> > >>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna <[email protected] > <mailto:[email protected]>> > >>>>> wrote: > >>>>>> > >>>>>> Hi All, > >>>>>> > >>>>>> Oozie is starting the release process for Oozie 4.3.0. This thread > >>>> is > >>>>> for discussion about the scope of the release. Please provide your > >>>>> suggestion about what should be included (or any exclusions) in the > >>>> release. > >>>>>> > >>>>>> Currently, we have around 158 patches submitted to trunk. Based on > >>>> the > >>>>> scope, we will include (exclude) more if required. > >>>>>> Version Upgrade > >>>>>> Java 1.7 (OOZIE-2036 < > >>>> https://issues.apache.org/jira/browse/OOZIE-2036 < > https://issues.apache.org/jira/browse/OOZIE-2036>>) > >>>>>> OpenJPA (OOZIE-2488 < > >>>> https://issues.apache.org/jira/browse/OOZIE-2488 < > https://issues.apache.org/jira/browse/OOZIE-2488>>), > >>>>>> Curator (OOZIE-2231 < > >>>> https://issues.apache.org/jira/browse/OOZIE-2231 < > https://issues.apache.org/jira/browse/OOZIE-2231> > >>>>>> ), > >>>>>> HttpClient (OOZIE-2538 < > >>>>> https://issues.apache.org/jira/browse/OOZIE-2538 < > https://issues.apache.org/jira/browse/OOZIE-2538>>), > >>>>>> Pig (OOZIE-2602 <https://issues.apache.org/jira/browse/OOZIE-2602 < > https://issues.apache.org/jira/browse/OOZIE-2602> > >>>>> ). > >>>>>> New Features/Improvements: Add default JT and NN, Database schema > >>>>> service check, instrumental metrics, Tomcat upgrade etc. > >>>>>> Bug Fixes. > >>>>>> Provide your opinion on inclusion/exclusion of the above points or > >>>> more > >>>>> for the scope of the release. We can also, take some work which is > >>>> already > >>>>> done and review is pending for them (depending on the availability > >>>> of the > >>>>> reviewers). > >>>>>> > >>>>>> Thanks, > >>>>>> Abhishek > >>>>> > >>>>> > >>>> > >>>> > >>>> > >> > > > >
