Rohini, that sounds good to me. Especially because we're on "master" branch but "trunk" version in JIRA and it gets a little confusing. Instead of deleting "trunk", I think we're better off just renaming it to the release number. Otherwise, we'll have to re-label any current "trunk" fields. We can use 4.4.0 for now, but I imagine that the next release might end up being 5.0.0 (with Oozie on Yarn); we can always rename it later.
I'm not sure if there's a specific rule here, but I suppose we should probably have a vote in the private list to make things official as this is a release-management-related change. Can you send an email? Abhishek, thanks for putting this all together. I think Rohini's suggestion of updating the JIRAs is the right next step. You're probably best off using the bulk change feature instead of doing each JIRA individually. I should have time to do a bunch of reviews this week, I've actually already started. I should have some time to continue doing more reviews. As a general thing, we've been pretty slow about getting these done :( - Robert On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy <[email protected]> wrote: > I also did mark some of the enhancement jiras that Satish was working on > and which might take time to review to 4.4.0 > > Regards, > Rohini > > On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy < > [email protected]> > wrote: > > > Abhishek, > > Can you mark all jiras already committed to trunk as 4.3.0 and those > > planned in the scope document as 4.3.0? This will help filter on those > > and review instead of referring to a google doc. I can pick up 10-15 > > jiras for review end of this week. We should try to get majority of > patches > > in the next 10 days and branch by Aug 12th. You can then use the 3rd week > > for any pending critical issues and rolling the release candidate. > > > > Robert, > > I am thinking, that we should get rid of the trunk version and use > > release numbers when committing from the beginning like all other > projects > > do. Unnecessary step to change from trunk to release version for every > > release. If folks are in agreement, then will delete the trunk version > > altogether. Does that require a mail to private? > > > > Regards, > > Rohini > > > > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <[email protected]> > > wrote: > > > >> Hi All, > >> > >> As we prepared a Oozie 4.3.0 scope document < > >> > https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu > > > >> and listed all the things which we wanted to include in the release. In > the > >> last few days, I spent some time and reviewed some of the patches and > >> provided my feedback. I will continue my efforts to review more patches. > >> These patches needs to be reviewed by committers and get committed. I > >> request other contributors also, review some of the patches and provide > >> their feedback. > >> > >> I am thinking, may be, we can decide upon some fixed (approximate) > >> release date and start working accordingly. Because, If we wait for all > the > >> listed items to be reviewed and committed, then it might take a lot more > >> time. I am thinking we should target third week of August to complete > and > >> release. > >> > >> Feel free to provide your feedback and suggestions. > >> > >> Thanks, > >> Abhishek > >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <[email protected]> > >> wrote: > >> > > >> > Hi, > >> > > >> > I looked into the available patches and prepared the list. I have put, > >> scope content which we have discussed till now, into a word document. > (For > >> the better formatting and easy to read and comment). > >> > > >> > I have noted the additional Jira into the list. Added two more Jira > >> related to spark (Scala version and Spark 2.0 property). > >> > > >> > Please take a look at the document (link below) and provide your > >> comments. > >> > > >> > If this looks good, we can start the release process. > >> > > >> > Scope Document: > >> > https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit# > >> > > >> > Thanks, > >> > Abhishek > >> > > >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <[email protected]> > >> wrote: > >> >> > >> >> That list looks good to me. The only one I'm not sure of is > OOZIE-2306 > >> >> (Oozie Health Check). It's a new feature and it might make sense to > >> let it > >> >> "bake" a while first? I'm also not sure it's quite ready yet. > >> >> > >> >> I'd also like to add a few others to the list: > >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie) > >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6) > >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2) > >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie) > >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other fixes) > >> >> > >> >> There's a number of other Patch Available JIRAs not on your list > >> (including > >> >> the ones I mentioned above). While we don't need to include > everything > >> >> here (and we probably shouldn't nor do we have the review bandwidth), > >> did > >> >> you consider each of them when creating the list? I currently see 97 > >> >> JIRAs: https://s.apache.org/oozie-patch-available > >> >> I think we should include all Spark Action fixes/improvements, given > >> how > >> >> popular it seems to be. > >> >> > >> >> thanks > >> >> - Robert > >> >> > >> >> > >> >> > >> >> > >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna < > [email protected]> > >> >> wrote: > >> >> > >> >>> Hi All, > >> >>> > >> >>> Below is the list of issues, picked from currently available > patches. > >> I > >> >>> could not decide up any 'open' issue, If there is anything, please > >> reply > >> >>> here. > >> >>> > >> >>> We can 'drop/add' more if required based on the feedback. Important > >> is, > >> >>> how many of these we will be able to review and commit. > >> >>> > >> >>> I will starting looking into these issues and provide feedback. > >> >>> > >> > --------------------------------------------------------------------------- > >> >>> OOZIE-1978 Forkjoin validation code is ridiculously slow in > some > >> cases > >> >>> OOZIE-2508 Documentation change for Coord action rerun > >> [OOZIE-1735] > >> >>> OOZIE-2583 oozie throws EL Exception when reference variable > name > >> >>> containing dot > >> >>> OOZIE-2582 Populating external child Ids for action failures > >> >>> OOZIE-2517 Add support for startCreatedTime and endCreatedTime > >> >>> filters for coord and bundles > >> >>> OOZIE-2512 ShareLibservice returns incorrect path for jar > >> >>> OOZIE-2572 SLA DURATION miss not shown when job is running for > >> longer > >> >>> than expected time > >> >>> OOZIE-2457 Oozie log parsing regex consume more than 90% cpu > >> >>> OOZIE-2574 Oozie to support replication-enabled mysql urls > >> >>> OOZIE-2506 Add logs into RecoverService for logging information > >> about > >> >>> queued commnads > >> >>> OOZIE-2573 dataset url which contains spaces can not be handled > >> >>> rightly > >> >>> OOZIE-2569 Adding yarn-site, core-site, hdfs-site and > mapred-site > >> >>> into spark launcher > >> >>> OOZIE-2491 oozie acl cannot specify group,it does`t work > >> >>> OOZIE-2539 Incorrect property key is used for 'hive log4j > >> >>> configuration file for execution mode' > >> >>> OOZIE-2555 Oozie SSL enable setup does not return port for > admin > >> >>> -servers > >> >>> OOZIE-2515 Duplicate information for "Changing > endtime/pausetime > >> of a > >> >>> Bundle Job" in CommandLineTool wiki > >> >>> OOZIE-2498 Oozie CallerId configuration for downstream > components > >> >>> OOZIE-2516 Update web service documentation for jobs API > >> >>> OOZIE-2500 -DtestJarSimple option mentioned in minioozie doc > >> does not > >> >>> work > >> >>> OOZIE-2513 log.scan.duration should not be used for error and > >> audit > >> >>> logs > >> >>> OOZIE-2461 Workflow, Coordinator and Bundle job querying should > >> have > >> >>> last modified filter > >> >>> OOZIE-2306 Oozie health check > >> >>> OOZIE-2518 Oozie is creating many files and directories in /tmp > >> per > >> >>> day > >> >>> OOZIE-2409 hive2 action with hive 1.2.1 failed > >> >>> OOZIE-2473 Connection pool for SMTP connection > >> >>> OOZIE-2440 Exponential re-try policy for workflow action > >> >>> OOZIE-2458 'oozie-setup.sh sharelib create' should ensure > >> uploaded > >> >>> jars are world readable > >> >>> OOZIE-2390 Rerun with failed option removing completed output > >> data > >> >>> OOZIE-1810 Workflow cannot get into Failed state when kill > >> control > >> >>> node cannot resolve variable in message > >> >>> OOZIE-2244 Oozie should mask passwords in the logs when logging > >> >>> command arguments > >> >>> > >> > --------------------------------------------------------------------------- > >> >>> Thanks, > >> >>> Abhishek > >> >>> > >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna <[email protected]> > >> >>> wrote: > >> >>>> > >> >>>> Hi All, > >> >>>> > >> >>>> I looked into the commits (approx. 160), all looked good to me and > >> >>> should be included into the release. If there are things, which > needs > >> to be > >> >>> excluded, provide your comments. > >> >>>> > >> >>>> We still need to decide on the patch which are in progress status. > >> >>>> > >> >>>> On the upgrade side: > >> >>>> Java version 1.6 to 1.7 > >> >>>> OpenJPA version 2.2.2 to 2.4.1 > >> >>>> Curator version 2.5.0 to 2.11.0 > >> >>>> Pig version 0.12.1 to 0.16.0 > >> >>>> Hive version 0.13.1 to 1.2.0 > >> >>>> HttpClient version 4.3 to 4.3.6 > >> >>>> commons-httpclient version 3.1 to 4.3.6 (discontinued) > >> >>>> > >> >>>> Please provide your feedback on the above upgrades. > >> >>>> > >> >>>> Once we decide on the scope of the release. We will start working > on > >> the > >> >>> completing/testing the release. > >> >>>> > >> >>>> Thanks, > >> >>>> Abhishek > >> >>>> > >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna <[email protected] > >> >>> <mailto:[email protected]>> wrote: > >> >>>>> > >> >>>>> Okay. We will make the Java version to 1.7. > >> >>>>> > >> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient, > Pig > >> etc. > >> >>>>> > >> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally as > >> well. > >> >>>>> > >> >>>>> As per the comments on the OOZIE-2231 < > >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator version > >> also > >> >>> can be upgraded to 2.11. > >> >>>>> > >> >>>>> Thanks, > >> >>>>> Abhishek > >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter <[email protected] > >> >>> <mailto:[email protected]>> wrote: > >> >>>>>> > >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for over a > >> year > >> >>> now > >> >>>>>> with no problems. We did have to comment out a few lines in > >> >>> MiniHCatServer > >> >>>>>> and MiniHS2 that were no longer needed: > >> >>>>>> > >> >>>>>> diff --git > >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >>>>>> index 8699ff8..d694977 100644 > >> >>>>>> --- > a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >>>>>> +++ > b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer { > >> >>>>>> public static void resetDefaultDBCreation() throws Exception { > >> >>>>>> // Need to do this, else default db will not be created for > >> >>> local > >> >>>>>> metastores. > >> >>>>>> // TestHiveMain will fail with > >> >>>>>> InvalidObjectException(message:There is no database named > default) > >> >>>>>> - Field declaredField = > >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >> >>>>>> - declaredField.setAccessible(true); > >> >>>>>> - declaredField.set(null, false); > >> >>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and > later > >> >>>>>> + //Field declaredField = > >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >> >>>>>> + //declaredField.setAccessible(true); > >> >>>>>> + //declaredField.set(null, false); > >> >>>>>> } > >> >>>>>> > >> >>>>>> public static void resetHiveConfStaticVariables() throws > >> Exception > >> >>> { > >> >>>>>> diff --git > >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >>>>>> index c9dfd56..505dc55 100644 > >> >>>>>> --- a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >>>>>> +++ b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends > AbstractHiveService > >> { > >> >>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, > >> getHost()); > >> >>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, > >> >>> getBinaryPort()); > >> >>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT, > >> >>>>>> getHttpPort()); > >> >>>>>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >> >>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and later > >> >>>>>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >> >>>>>> > >> >>>>>> Path scratchDir = new Path(baseDfsDir, "scratch"); > >> >>>>>> fs.mkdirs(scratchDir); > >> >>>>>> > >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan < > >> >>>>>> [email protected] <mailto: > [email protected] > >> >> > >> >>> wrote: > >> >>>>>> > >> >>>>>>> Let us explore changing the default hive version also to 1.x. > >> 0.13 > >> >>> is > >> >>>>>>> old > >> >>>>>>> > >> >>>>>>> Thanks > >> >>>>>>> > >> >>>>>>> Venkat > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <[email protected] > >> <mailto: > >> >>> [email protected]>> wrote: > >> >>>>>>> > >> >>>>>>> I don't think we have any guidelines defined anywhere about > when > >> >>> it's > >> >>>>>>> okay > >> >>>>>>> to change the minimum Java version, but perhaps we need to wait > >> >>> until > >> >>>>>>> the > >> >>>>>>> next major release (5.0)? That said, Hadoop did this in a > minor > >> >>>>>>> release > >> >>>>>>> and Java 6 was EoL in 2013. So I'm fine with doing it in 4.3 > if > >> >>>>>>> nobody has > >> >>>>>>> an objections; we should just make sure to call that out in our > >> >>> release > >> >>>>>>> notes. > >> >>>>>>> > >> >>>>>>> Otherwise, nothing comes to mind for anything we should leave > out > >> >>> other > >> >>>>>>> than the Oozie on Yarn stuff, which is not currently in trunk > >> >>> anyway. > >> >>>>>>> > >> >>>>>>> I know there's a number of Spark Action JIRAs; we should make > >> sure > >> >>> to > >> >>>>>>> include all of these and any pending ones. > >> >>>>>>> @Rohini, @Puru, @Satish, anything you think needs more time to > >> bake > >> >>>>>>> that we > >> >>>>>>> should hold off on including in the release? > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> thanks > >> >>>>>>> - Robert > >> >>>>>>> > >> >>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna < > >> >>> [email protected] <mailto:[email protected]>> > >> >>>>>>> wrote: > >> >>>>>>> > >> >>>>>>>> Hi All, > >> >>>>>>>> > >> >>>>>>>> Any suggestions or comments on the scope for the release? > >> >>>>>>>> > >> >>>>>>>> Thanks, > >> >>>>>>>> Abhishek > >> >>>>>>>> > >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna < > >> [email protected] > >> >>> <mailto:[email protected]>> > >> >>>>>>>> wrote: > >> >>>>>>>>> > >> >>>>>>>>> Hi All, > >> >>>>>>>>> > >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This > >> thread > >> >>>>>>> is > >> >>>>>>>> for discussion about the scope of the release. Please provide > >> your > >> >>>>>>>> suggestion about what should be included (or any exclusions) in > >> the > >> >>>>>>> release. > >> >>>>>>>>> > >> >>>>>>>>> Currently, we have around 158 patches submitted to trunk. > Based > >> on > >> >>>>>>> the > >> >>>>>>>> scope, we will include (exclude) more if required. > >> >>>>>>>>> Version Upgrade > >> >>>>>>>>> Java 1.7 (OOZIE-2036 < > >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 < > >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>) > >> >>>>>>>>> OpenJPA (OOZIE-2488 < > >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 < > >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>), > >> >>>>>>>>> Curator (OOZIE-2231 < > >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 < > >> >>> https://issues.apache.org/jira/browse/OOZIE-2231> > >> >>>>>>>>> ), > >> >>>>>>>>> HttpClient (OOZIE-2538 < > >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 < > >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>), > >> >>>>>>>>> Pig (OOZIE-2602 < > >> https://issues.apache.org/jira/browse/OOZIE-2602 < > >> >>> https://issues.apache.org/jira/browse/OOZIE-2602> > >> >>>>>>>> ). > >> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database > >> schema > >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc. > >> >>>>>>>>> Bug Fixes. > >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above > points > >> or > >> >>>>>>> more > >> >>>>>>>> for the scope of the release. We can also, take some work which > >> is > >> >>>>>>> already > >> >>>>>>>> done and review is pending for them (depending on the > >> availability > >> >>>>>>> of the > >> >>>>>>>> reviewers). > >> >>>>>>>>> > >> >>>>>>>>> Thanks, > >> >>>>>>>>> Abhishek > >> >>>>>>>> > >> >>>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>>>> > >> >>>>> > >> >>>> > >> >>> > >> >>> > >> > > >> > >> > > >
