I also did mark some of the enhancement jiras that Satish was working on and which might take time to review to 4.4.0
Regards, Rohini On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy <[email protected]> wrote: > Abhishek, > Can you mark all jiras already committed to trunk as 4.3.0 and those > planned in the scope document as 4.3.0? This will help filter on those > and review instead of referring to a google doc. I can pick up 10-15 > jiras for review end of this week. We should try to get majority of patches > in the next 10 days and branch by Aug 12th. You can then use the 3rd week > for any pending critical issues and rolling the release candidate. > > Robert, > I am thinking, that we should get rid of the trunk version and use > release numbers when committing from the beginning like all other projects > do. Unnecessary step to change from trunk to release version for every > release. If folks are in agreement, then will delete the trunk version > altogether. Does that require a mail to private? > > Regards, > Rohini > > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <[email protected]> > wrote: > >> Hi All, >> >> As we prepared a Oozie 4.3.0 scope document < >> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu> >> and listed all the things which we wanted to include in the release. In the >> last few days, I spent some time and reviewed some of the patches and >> provided my feedback. I will continue my efforts to review more patches. >> These patches needs to be reviewed by committers and get committed. I >> request other contributors also, review some of the patches and provide >> their feedback. >> >> I am thinking, may be, we can decide upon some fixed (approximate) >> release date and start working accordingly. Because, If we wait for all the >> listed items to be reviewed and committed, then it might take a lot more >> time. I am thinking we should target third week of August to complete and >> release. >> >> Feel free to provide your feedback and suggestions. >> >> Thanks, >> Abhishek >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <[email protected]> >> wrote: >> > >> > Hi, >> > >> > I looked into the available patches and prepared the list. I have put, >> scope content which we have discussed till now, into a word document. (For >> the better formatting and easy to read and comment). >> > >> > I have noted the additional Jira into the list. Added two more Jira >> related to spark (Scala version and Spark 2.0 property). >> > >> > Please take a look at the document (link below) and provide your >> comments. >> > >> > If this looks good, we can start the release process. >> > >> > Scope Document: >> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit# >> > >> > Thanks, >> > Abhishek >> > >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <[email protected]> >> wrote: >> >> >> >> That list looks good to me. The only one I'm not sure of is OOZIE-2306 >> >> (Oozie Health Check). It's a new feature and it might make sense to >> let it >> >> "bake" a while first? I'm also not sure it's quite ready yet. >> >> >> >> I'd also like to add a few others to the list: >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie) >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6) >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2) >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie) >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes) >> >> >> >> There's a number of other Patch Available JIRAs not on your list >> (including >> >> the ones I mentioned above). While we don't need to include everything >> >> here (and we probably shouldn't nor do we have the review bandwidth), >> did >> >> you consider each of them when creating the list? I currently see 97 >> >> JIRAs: https://s.apache.org/oozie-patch-available >> >> I think we should include all Spark Action fixes/improvements, given >> how >> >> popular it seems to be. >> >> >> >> thanks >> >> - Robert >> >> >> >> >> >> >> >> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna <[email protected]> >> >> wrote: >> >> >> >>> Hi All, >> >>> >> >>> Below is the list of issues, picked from currently available patches. >> I >> >>> could not decide up any 'open' issue, If there is anything, please >> reply >> >>> here. >> >>> >> >>> We can 'drop/add' more if required based on the feedback. Important >> is, >> >>> how many of these we will be able to review and commit. >> >>> >> >>> I will starting looking into these issues and provide feedback. >> >>> >> --------------------------------------------------------------------------- >> >>> OOZIE-1978 Forkjoin validation code is ridiculously slow in some >> cases >> >>> OOZIE-2508 Documentation change for Coord action rerun >> [OOZIE-1735] >> >>> OOZIE-2583 oozie throws EL Exception when reference variable name >> >>> containing dot >> >>> OOZIE-2582 Populating external child Ids for action failures >> >>> OOZIE-2517 Add support for startCreatedTime and endCreatedTime >> >>> filters for coord and bundles >> >>> OOZIE-2512 ShareLibservice returns incorrect path for jar >> >>> OOZIE-2572 SLA DURATION miss not shown when job is running for >> longer >> >>> than expected time >> >>> OOZIE-2457 Oozie log parsing regex consume more than 90% cpu >> >>> OOZIE-2574 Oozie to support replication-enabled mysql urls >> >>> OOZIE-2506 Add logs into RecoverService for logging information >> about >> >>> queued commnads >> >>> OOZIE-2573 dataset url which contains spaces can not be handled >> >>> rightly >> >>> OOZIE-2569 Adding yarn-site, core-site, hdfs-site and mapred-site >> >>> into spark launcher >> >>> OOZIE-2491 oozie acl cannot specify group,it does`t work >> >>> OOZIE-2539 Incorrect property key is used for 'hive log4j >> >>> configuration file for execution mode' >> >>> OOZIE-2555 Oozie SSL enable setup does not return port for admin >> >>> -servers >> >>> OOZIE-2515 Duplicate information for "Changing endtime/pausetime >> of a >> >>> Bundle Job" in CommandLineTool wiki >> >>> OOZIE-2498 Oozie CallerId configuration for downstream components >> >>> OOZIE-2516 Update web service documentation for jobs API >> >>> OOZIE-2500 -DtestJarSimple option mentioned in minioozie doc >> does not >> >>> work >> >>> OOZIE-2513 log.scan.duration should not be used for error and >> audit >> >>> logs >> >>> OOZIE-2461 Workflow, Coordinator and Bundle job querying should >> have >> >>> last modified filter >> >>> OOZIE-2306 Oozie health check >> >>> OOZIE-2518 Oozie is creating many files and directories in /tmp >> per >> >>> day >> >>> OOZIE-2409 hive2 action with hive 1.2.1 failed >> >>> OOZIE-2473 Connection pool for SMTP connection >> >>> OOZIE-2440 Exponential re-try policy for workflow action >> >>> OOZIE-2458 'oozie-setup.sh sharelib create' should ensure >> uploaded >> >>> jars are world readable >> >>> OOZIE-2390 Rerun with failed option removing completed output >> data >> >>> OOZIE-1810 Workflow cannot get into Failed state when kill >> control >> >>> node cannot resolve variable in message >> >>> OOZIE-2244 Oozie should mask passwords in the logs when logging >> >>> command arguments >> >>> >> --------------------------------------------------------------------------- >> >>> Thanks, >> >>> Abhishek >> >>> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <[email protected]> >> >>> wrote: >> >>>> >> >>>> Hi All, >> >>>> >> >>>> I looked into the commits (approx. 160), all looked good to me and >> >>> should be included into the release. If there are things, which needs >> to be >> >>> excluded, provide your comments. >> >>>> >> >>>> We still need to decide on the patch which are in progress status. >> >>>> >> >>>> On the upgrade side: >> >>>> Java version 1.6 to 1.7 >> >>>> OpenJPA version 2.2.2 to 2.4.1 >> >>>> Curator version 2.5.0 to 2.11.0 >> >>>> Pig version 0.12.1 to 0.16.0 >> >>>> Hive version 0.13.1 to 1.2.0 >> >>>> HttpClient version 4.3 to 4.3.6 >> >>>> commons-httpclient version 3.1 to 4.3.6 (discontinued) >> >>>> >> >>>> Please provide your feedback on the above upgrades. >> >>>> >> >>>> Once we decide on the scope of the release. We will start working on >> the >> >>> completing/testing the release. >> >>>> >> >>>> Thanks, >> >>>> Abhishek >> >>>> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <[email protected] >> >>> <mailto:[email protected]>> wrote: >> >>>>> >> >>>>> Okay. We will make the Java version to 1.7. >> >>>>> >> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient, Pig >> etc. >> >>>>> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally as >> well. >> >>>>> >> >>>>> As per the comments on the OOZIE-2231 < >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version >> also >> >>> can be upgraded to 2.11. >> >>>>> >> >>>>> Thanks, >> >>>>> Abhishek >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <[email protected] >> >>> <mailto:[email protected]>> wrote: >> >>>>>> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over a >> year >> >>> now >> >>>>>> with no problems. We did have to comment out a few lines in >> >>> MiniHCatServer >> >>>>>> and MiniHS2 that were no longer needed: >> >>>>>> >> >>>>>> diff --git >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >> >>>>>> index 8699ff8..d694977 100644 >> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer { >> >>>>>> public static void resetDefaultDBCreation() throws Exception { >> >>>>>> // Need to do this, else default db will not be created for >> >>> local >> >>>>>> metastores. >> >>>>>> // TestHiveMain will fail with >> >>>>>> InvalidObjectException(message:There is no database named default) >> >>>>>> - Field declaredField = >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); >> >>>>>> - declaredField.setAccessible(true); >> >>>>>> - declaredField.set(null, false); >> >>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later >> >>>>>> + //Field declaredField = >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); >> >>>>>> + //declaredField.setAccessible(true); >> >>>>>> + //declaredField.set(null, false); >> >>>>>> } >> >>>>>> >> >>>>>> public static void resetHiveConfStaticVariables() throws >> Exception >> >>> { >> >>>>>> diff --git >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >> >>>>>> index c9dfd56..505dc55 100644 >> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends AbstractHiveService >> { >> >>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, >> getHost()); >> >>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, >> >>> getBinaryPort()); >> >>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT, >> >>>>>> getHttpPort()); >> >>>>>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag(); >> >>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later >> >>>>>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag(); >> >>>>>> >> >>>>>> Path scratchDir = new Path(baseDfsDir, "scratch"); >> >>>>>> fs.mkdirs(scratchDir); >> >>>>>> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan < >> >>>>>> [email protected] <mailto:[email protected] >> >> >> >>> wrote: >> >>>>>> >> >>>>>>> Let us explore changing the default hive version also to 1.x. >> 0.13 >> >>> is >> >>>>>>> old >> >>>>>>> >> >>>>>>> Thanks >> >>>>>>> >> >>>>>>> Venkat >> >>>>>>> >> >>>>>>> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <[email protected] >> <mailto: >> >>> [email protected]>> wrote: >> >>>>>>> >> >>>>>>> I don't think we have any guidelines defined anywhere about when >> >>> it's >> >>>>>>> okay >> >>>>>>> to change the minimum Java version, but perhaps we need to wait >> >>> until >> >>>>>>> the >> >>>>>>> next major release (5.0)? That said, Hadoop did this in a minor >> >>>>>>> release >> >>>>>>> and Java 6 was EoL in 2013. So I'm fine with doing it in 4.3 if >> >>>>>>> nobody has >> >>>>>>> an objections; we should just make sure to call that out in our >> >>> release >> >>>>>>> notes. >> >>>>>>> >> >>>>>>> Otherwise, nothing comes to mind for anything we should leave out >> >>> other >> >>>>>>> than the Oozie on Yarn stuff, which is not currently in trunk >> >>> anyway. >> >>>>>>> >> >>>>>>> I know there's a number of Spark Action JIRAs; we should make >> sure >> >>> to >> >>>>>>> include all of these and any pending ones. >> >>>>>>> @Rohini, @Puru, @Satish, anything you think needs more time to >> bake >> >>>>>>> that we >> >>>>>>> should hold off on including in the release? >> >>>>>>> >> >>>>>>> >> >>>>>>> thanks >> >>>>>>> - Robert >> >>>>>>> >> >>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna < >> >>> [email protected] <mailto:[email protected]>> >> >>>>>>> wrote: >> >>>>>>> >> >>>>>>>> Hi All, >> >>>>>>>> >> >>>>>>>> Any suggestions or comments on the scope for the release? >> >>>>>>>> >> >>>>>>>> Thanks, >> >>>>>>>> Abhishek >> >>>>>>>> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna < >> [email protected] >> >>> <mailto:[email protected]>> >> >>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>> Hi All, >> >>>>>>>>> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This >> thread >> >>>>>>> is >> >>>>>>>> for discussion about the scope of the release. Please provide >> your >> >>>>>>>> suggestion about what should be included (or any exclusions) in >> the >> >>>>>>> release. >> >>>>>>>>> >> >>>>>>>>> Currently, we have around 158 patches submitted to trunk. Based >> on >> >>>>>>> the >> >>>>>>>> scope, we will include (exclude) more if required. >> >>>>>>>>> Version Upgrade >> >>>>>>>>> Java 1.7 (OOZIE-2036 < >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 < >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>) >> >>>>>>>>> OpenJPA (OOZIE-2488 < >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 < >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>), >> >>>>>>>>> Curator (OOZIE-2231 < >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 < >> >>> https://issues.apache.org/jira/browse/OOZIE-2231> >> >>>>>>>>> ), >> >>>>>>>>> HttpClient (OOZIE-2538 < >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 < >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>), >> >>>>>>>>> Pig (OOZIE-2602 < >> https://issues.apache.org/jira/browse/OOZIE-2602 < >> >>> https://issues.apache.org/jira/browse/OOZIE-2602> >> >>>>>>>> ). >> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database >> schema >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc. >> >>>>>>>>> Bug Fixes. >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above points >> or >> >>>>>>> more >> >>>>>>>> for the scope of the release. We can also, take some work which >> is >> >>>>>>> already >> >>>>>>>> done and review is pending for them (depending on the >> availability >> >>>>>>> of the >> >>>>>>>> reviewers). >> >>>>>>>>> >> >>>>>>>>> Thanks, >> >>>>>>>>> Abhishek >> >>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>>>> >> >>>>> >> >>>> >> >>> >> >>> >> > >> >> >
