I think it would be better to send out email for the bulk change. If there are any issues or jiras wrongly marked someone can respond. We do that with Pig and there are always couple of jiras that folks come back with feedback on pulling back to release.
On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter <rkan...@cloudera.com> wrote: > Abhishek, next time you do a bulk change, please remember to tell it not to > send out an email for every change :) > > On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter <rkan...@cloudera.com> > wrote: > > > Rohini, that sounds good to me. Especially because we're on "master" > > branch but "trunk" version in JIRA and it gets a little confusing. > Instead > > of deleting "trunk", I think we're better off just renaming it to the > > release number. Otherwise, we'll have to re-label any current "trunk" > > fields. We can use 4.4.0 for now, but I imagine that the next release > > might end up being 5.0.0 (with Oozie on Yarn); we can always rename it > > later. > > > > I'm not sure if there's a specific rule here, but I suppose we should > > probably have a vote in the private list to make things official as this > is > > a release-management-related change. Can you send an email? > > > > Abhishek, thanks for putting this all together. I think Rohini's > > suggestion of updating the JIRAs is the right next step. You're probably > > best off using the bulk change feature instead of doing each JIRA > > individually. I should have time to do a bunch of reviews this week, > I've > > actually already started. I should have some time to continue doing more > > reviews. As a general thing, we've been pretty slow about getting these > > done :( > > > > - Robert > > > > On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy < > > rohini.adi...@gmail.com> wrote: > > > >> I also did mark some of the enhancement jiras that Satish was working on > >> and which might take time to review to 4.4.0 > >> > >> Regards, > >> Rohini > >> > >> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy < > >> rohini.adi...@gmail.com> > >> wrote: > >> > >> > Abhishek, > >> > Can you mark all jiras already committed to trunk as 4.3.0 and > those > >> > planned in the scope document as 4.3.0? This will help filter on those > >> > and review instead of referring to a google doc. I can pick up 10-15 > >> > jiras for review end of this week. We should try to get majority of > >> patches > >> > in the next 10 days and branch by Aug 12th. You can then use the 3rd > >> week > >> > for any pending critical issues and rolling the release candidate. > >> > > >> > Robert, > >> > I am thinking, that we should get rid of the trunk version and use > >> > release numbers when committing from the beginning like all other > >> projects > >> > do. Unnecessary step to change from trunk to release version for every > >> > release. If folks are in agreement, then will delete the trunk version > >> > altogether. Does that require a mail to private? > >> > > >> > Regards, > >> > Rohini > >> > > >> > On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna <bafna.i...@gmail.com > > > >> > wrote: > >> > > >> >> Hi All, > >> >> > >> >> As we prepared a Oozie 4.3.0 scope document < > >> >> > >> > https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu > >> > > >> >> and listed all the things which we wanted to include in the release. > >> In the > >> >> last few days, I spent some time and reviewed some of the patches and > >> >> provided my feedback. I will continue my efforts to review more > >> patches. > >> >> These patches needs to be reviewed by committers and get committed. I > >> >> request other contributors also, review some of the patches and > provide > >> >> their feedback. > >> >> > >> >> I am thinking, may be, we can decide upon some fixed (approximate) > >> >> release date and start working accordingly. Because, If we wait for > >> all the > >> >> listed items to be reviewed and committed, then it might take a lot > >> more > >> >> time. I am thinking we should target third week of August to complete > >> and > >> >> release. > >> >> > >> >> Feel free to provide your feedback and suggestions. > >> >> > >> >> Thanks, > >> >> Abhishek > >> >> > On Jul 21, 2016, at 4:39 PM, Abhishek Bafna <bafna.i...@gmail.com> > >> >> wrote: > >> >> > > >> >> > Hi, > >> >> > > >> >> > I looked into the available patches and prepared the list. I have > >> put, > >> >> scope content which we have discussed till now, into a word document. > >> (For > >> >> the better formatting and easy to read and comment). > >> >> > > >> >> > I have noted the additional Jira into the list. Added two more Jira > >> >> related to spark (Scala version and Spark 2.0 property). > >> >> > > >> >> > Please take a look at the document (link below) and provide your > >> >> comments. > >> >> > > >> >> > If this looks good, we can start the release process. > >> >> > > >> >> > Scope Document: > >> >> > >> > https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5GmFsuHQZmrT63Ow/edit# > >> >> > > >> >> > Thanks, > >> >> > Abhishek > >> >> > > >> >> >> On Jul 21, 2016, at 5:07 AM, Robert Kanter <rkan...@cloudera.com> > >> >> wrote: > >> >> >> > >> >> >> That list looks good to me. The only one I'm not sure of is > >> OOZIE-2306 > >> >> >> (Oozie Health Check). It's a new feature and it might make sense > to > >> >> let it > >> >> >> "bake" a while first? I'm also not sure it's quite ready yet. > >> >> >> > >> >> >> I'd also like to add a few others to the list: > >> >> >> - OOZIE-2273 (MiniOozie does not work outside of Oozie) > >> >> >> - The subtasks under OOZIE-2036 (Drop support for Java 1.6) > >> >> >> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2) > >> >> >> --- OOZIE-1793 (Improve find bugs reporting for Oozie) > >> >> >> --- OOZIE-2552 (Update ActiveMQ version for security and other > >> fixes) > >> >> >> > >> >> >> There's a number of other Patch Available JIRAs not on your list > >> >> (including > >> >> >> the ones I mentioned above). While we don't need to include > >> everything > >> >> >> here (and we probably shouldn't nor do we have the review > >> bandwidth), > >> >> did > >> >> >> you consider each of them when creating the list? I currently see > >> 97 > >> >> >> JIRAs: https://s.apache.org/oozie-patch-available > >> >> >> I think we should include all Spark Action fixes/improvements, > given > >> >> how > >> >> >> popular it seems to be. > >> >> >> > >> >> >> thanks > >> >> >> - Robert > >> >> >> > >> >> >> > >> >> >> > >> >> >> > >> >> >> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna < > >> bafna.i...@gmail.com> > >> >> >> wrote: > >> >> >> > >> >> >>> Hi All, > >> >> >>> > >> >> >>> Below is the list of issues, picked from currently available > >> patches. > >> >> I > >> >> >>> could not decide up any 'open' issue, If there is anything, > please > >> >> reply > >> >> >>> here. > >> >> >>> > >> >> >>> We can 'drop/add' more if required based on the feedback. > Important > >> >> is, > >> >> >>> how many of these we will be able to review and commit. > >> >> >>> > >> >> >>> I will starting looking into these issues and provide feedback. > >> >> >>> > >> >> > >> > --------------------------------------------------------------------------- > >> >> >>> OOZIE-1978 Forkjoin validation code is ridiculously slow in > >> some > >> >> cases > >> >> >>> OOZIE-2508 Documentation change for Coord action rerun > >> >> [OOZIE-1735] > >> >> >>> OOZIE-2583 oozie throws EL Exception when reference variable > >> name > >> >> >>> containing dot > >> >> >>> OOZIE-2582 Populating external child Ids for action failures > >> >> >>> OOZIE-2517 Add support for startCreatedTime and > endCreatedTime > >> >> >>> filters for coord and bundles > >> >> >>> OOZIE-2512 ShareLibservice returns incorrect path for jar > >> >> >>> OOZIE-2572 SLA DURATION miss not shown when job is running > for > >> >> longer > >> >> >>> than expected time > >> >> >>> OOZIE-2457 Oozie log parsing regex consume more than 90% cpu > >> >> >>> OOZIE-2574 Oozie to support replication-enabled mysql urls > >> >> >>> OOZIE-2506 Add logs into RecoverService for logging > >> information > >> >> about > >> >> >>> queued commnads > >> >> >>> OOZIE-2573 dataset url which contains spaces can not be > >> handled > >> >> >>> rightly > >> >> >>> OOZIE-2569 Adding yarn-site, core-site, hdfs-site and > >> mapred-site > >> >> >>> into spark launcher > >> >> >>> OOZIE-2491 oozie acl cannot specify group,it does`t work > >> >> >>> OOZIE-2539 Incorrect property key is used for 'hive log4j > >> >> >>> configuration file for execution mode' > >> >> >>> OOZIE-2555 Oozie SSL enable setup does not return port for > >> admin > >> >> >>> -servers > >> >> >>> OOZIE-2515 Duplicate information for "Changing > >> endtime/pausetime > >> >> of a > >> >> >>> Bundle Job" in CommandLineTool wiki > >> >> >>> OOZIE-2498 Oozie CallerId configuration for downstream > >> components > >> >> >>> OOZIE-2516 Update web service documentation for jobs API > >> >> >>> OOZIE-2500 -DtestJarSimple option mentioned in minioozie doc > >> >> does not > >> >> >>> work > >> >> >>> OOZIE-2513 log.scan.duration should not be used for error > and > >> >> audit > >> >> >>> logs > >> >> >>> OOZIE-2461 Workflow, Coordinator and Bundle job querying > >> should > >> >> have > >> >> >>> last modified filter > >> >> >>> OOZIE-2306 Oozie health check > >> >> >>> OOZIE-2518 Oozie is creating many files and directories in > >> /tmp > >> >> per > >> >> >>> day > >> >> >>> OOZIE-2409 hive2 action with hive 1.2.1 failed > >> >> >>> OOZIE-2473 Connection pool for SMTP connection > >> >> >>> OOZIE-2440 Exponential re-try policy for workflow action > >> >> >>> OOZIE-2458 'oozie-setup.sh sharelib create' should ensure > >> >> uploaded > >> >> >>> jars are world readable > >> >> >>> OOZIE-2390 Rerun with failed option removing completed > output > >> >> data > >> >> >>> OOZIE-1810 Workflow cannot get into Failed state when kill > >> >> control > >> >> >>> node cannot resolve variable in message > >> >> >>> OOZIE-2244 Oozie should mask passwords in the logs when > >> logging > >> >> >>> command arguments > >> >> >>> > >> >> > >> > --------------------------------------------------------------------------- > >> >> >>> Thanks, > >> >> >>> Abhishek > >> >> >>> > >> >> >>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna < > bafna.i...@gmail.com > >> > > >> >> >>> wrote: > >> >> >>>> > >> >> >>>> Hi All, > >> >> >>>> > >> >> >>>> I looked into the commits (approx. 160), all looked good to me > and > >> >> >>> should be included into the release. If there are things, which > >> needs > >> >> to be > >> >> >>> excluded, provide your comments. > >> >> >>>> > >> >> >>>> We still need to decide on the patch which are in progress > status. > >> >> >>>> > >> >> >>>> On the upgrade side: > >> >> >>>> Java version 1.6 to 1.7 > >> >> >>>> OpenJPA version 2.2.2 to 2.4.1 > >> >> >>>> Curator version 2.5.0 to 2.11.0 > >> >> >>>> Pig version 0.12.1 to 0.16.0 > >> >> >>>> Hive version 0.13.1 to 1.2.0 > >> >> >>>> HttpClient version 4.3 to 4.3.6 > >> >> >>>> commons-httpclient version 3.1 to 4.3.6 (discontinued) > >> >> >>>> > >> >> >>>> Please provide your feedback on the above upgrades. > >> >> >>>> > >> >> >>>> Once we decide on the scope of the release. We will start > working > >> on > >> >> the > >> >> >>> completing/testing the release. > >> >> >>>> > >> >> >>>> Thanks, > >> >> >>>> Abhishek > >> >> >>>> > >> >> >>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna < > >> bafna.i...@gmail.com > >> >> >>> <mailto:bafna.i...@gmail.com>> wrote: > >> >> >>>>> > >> >> >>>>> Okay. We will make the Java version to 1.7. > >> >> >>>>> > >> >> >>>>> There are upgrades we can do like OpenJPA, Curator, HttpClient, > >> Pig > >> >> etc. > >> >> >>>>> > >> >> >>>>> HttpClient, Pig (0.16) version upgrade we have done internally > as > >> >> well. > >> >> >>>>> > >> >> >>>>> As per the comments on the OOZIE-2231 < > >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231>, curator > version > >> >> also > >> >> >>> can be upgraded to 2.11. > >> >> >>>>> > >> >> >>>>> Thanks, > >> >> >>>>> Abhishek > >> >> >>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter < > >> rkan...@cloudera.com > >> >> >>> <mailto:rkan...@cloudera.com>> wrote: > >> >> >>>>>> > >> >> >>>>>> We've been using a version of Hive/HCat based on 1.1.0 for > over > >> a > >> >> year > >> >> >>> now > >> >> >>>>>> with no problems. We did have to comment out a few lines in > >> >> >>> MiniHCatServer > >> >> >>>>>> and MiniHS2 that were no longer needed: > >> >> >>>>>> > >> >> >>>>>> diff --git > >> >> >>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >> >>>>>> index 8699ff8..d694977 100644 > >> >> >>>>>> --- > >> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >> >>>>>> +++ > >> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java > >> >> >>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer { > >> >> >>>>>> public static void resetDefaultDBCreation() throws > Exception { > >> >> >>>>>> // Need to do this, else default db will not be created > >> for > >> >> >>> local > >> >> >>>>>> metastores. > >> >> >>>>>> // TestHiveMain will fail with > >> >> >>>>>> InvalidObjectException(message:There is no database named > >> default) > >> >> >>>>>> - Field declaredField = > >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >> >> >>>>>> - declaredField.setAccessible(true); > >> >> >>>>>> - declaredField.set(null, false); > >> >> >>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and > >> later > >> >> >>>>>> + //Field declaredField = > >> >> >>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); > >> >> >>>>>> + //declaredField.setAccessible(true); > >> >> >>>>>> + //declaredField.set(null, false); > >> >> >>>>>> } > >> >> >>>>>> > >> >> >>>>>> public static void resetHiveConfStaticVariables() throws > >> >> Exception > >> >> >>> { > >> >> >>>>>> diff --git > >> >> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >> >>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >> >>>>>> index c9dfd56..505dc55 100644 > >> >> >>>>>> --- > a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >> >>>>>> +++ > b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java > >> >> >>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends > >> AbstractHiveService > >> >> { > >> >> >>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, > >> >> getHost()); > >> >> >>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_PORT, > >> >> >>> getBinaryPort()); > >> >> >>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT, > >> >> >>>>>> getHttpPort()); > >> >> >>>>>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >> >> >>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 and > later > >> >> >>>>>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag(); > >> >> >>>>>> > >> >> >>>>>> Path scratchDir = new Path(baseDfsDir, "scratch"); > >> >> >>>>>> fs.mkdirs(scratchDir); > >> >> >>>>>> > >> >> >>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan < > >> >> >>>>>> vranganat...@hortonworks.com <mailto: > >> vranganat...@hortonworks.com > >> >> >> > >> >> >>> wrote: > >> >> >>>>>> > >> >> >>>>>>> Let us explore changing the default hive version also to > 1.x. > >> >> 0.13 > >> >> >>> is > >> >> >>>>>>> old > >> >> >>>>>>> > >> >> >>>>>>> Thanks > >> >> >>>>>>> > >> >> >>>>>>> Venkat > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" <rkan...@cloudera.com > >> >> <mailto: > >> >> >>> rkan...@cloudera.com>> wrote: > >> >> >>>>>>> > >> >> >>>>>>> I don't think we have any guidelines defined anywhere about > >> when > >> >> >>> it's > >> >> >>>>>>> okay > >> >> >>>>>>> to change the minimum Java version, but perhaps we need to > >> wait > >> >> >>> until > >> >> >>>>>>> the > >> >> >>>>>>> next major release (5.0)? That said, Hadoop did this in a > >> minor > >> >> >>>>>>> release > >> >> >>>>>>> and Java 6 was EoL in 2013. So I'm fine with doing it in > 4.3 > >> if > >> >> >>>>>>> nobody has > >> >> >>>>>>> an objections; we should just make sure to call that out in > >> our > >> >> >>> release > >> >> >>>>>>> notes. > >> >> >>>>>>> > >> >> >>>>>>> Otherwise, nothing comes to mind for anything we should > leave > >> out > >> >> >>> other > >> >> >>>>>>> than the Oozie on Yarn stuff, which is not currently in > trunk > >> >> >>> anyway. > >> >> >>>>>>> > >> >> >>>>>>> I know there's a number of Spark Action JIRAs; we should > make > >> >> sure > >> >> >>> to > >> >> >>>>>>> include all of these and any pending ones. > >> >> >>>>>>> @Rohini, @Puru, @Satish, anything you think needs more time > to > >> >> bake > >> >> >>>>>>> that we > >> >> >>>>>>> should hold off on including in the release? > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> thanks > >> >> >>>>>>> - Robert > >> >> >>>>>>> > >> >> >>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna < > >> >> >>> bafna.i...@gmail.com <mailto:bafna.i...@gmail.com>> > >> >> >>>>>>> wrote: > >> >> >>>>>>> > >> >> >>>>>>>> Hi All, > >> >> >>>>>>>> > >> >> >>>>>>>> Any suggestions or comments on the scope for the release? > >> >> >>>>>>>> > >> >> >>>>>>>> Thanks, > >> >> >>>>>>>> Abhishek > >> >> >>>>>>>> > >> >> >>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna < > >> >> bafna.i...@gmail.com > >> >> >>> <mailto:bafna.i...@gmail.com>> > >> >> >>>>>>>> wrote: > >> >> >>>>>>>>> > >> >> >>>>>>>>> Hi All, > >> >> >>>>>>>>> > >> >> >>>>>>>>> Oozie is starting the release process for Oozie 4.3.0. This > >> >> thread > >> >> >>>>>>> is > >> >> >>>>>>>> for discussion about the scope of the release. Please > provide > >> >> your > >> >> >>>>>>>> suggestion about what should be included (or any exclusions) > >> in > >> >> the > >> >> >>>>>>> release. > >> >> >>>>>>>>> > >> >> >>>>>>>>> Currently, we have around 158 patches submitted to trunk. > >> Based > >> >> on > >> >> >>>>>>> the > >> >> >>>>>>>> scope, we will include (exclude) more if required. > >> >> >>>>>>>>> Version Upgrade > >> >> >>>>>>>>> Java 1.7 (OOZIE-2036 < > >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 < > >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2036>>) > >> >> >>>>>>>>> OpenJPA (OOZIE-2488 < > >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 < > >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2488>>), > >> >> >>>>>>>>> Curator (OOZIE-2231 < > >> >> >>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 < > >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2231> > >> >> >>>>>>>>> ), > >> >> >>>>>>>>> HttpClient (OOZIE-2538 < > >> >> >>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 < > >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2538>>), > >> >> >>>>>>>>> Pig (OOZIE-2602 < > >> >> https://issues.apache.org/jira/browse/OOZIE-2602 < > >> >> >>> https://issues.apache.org/jira/browse/OOZIE-2602> > >> >> >>>>>>>> ). > >> >> >>>>>>>>> New Features/Improvements: Add default JT and NN, Database > >> >> schema > >> >> >>>>>>>> service check, instrumental metrics, Tomcat upgrade etc. > >> >> >>>>>>>>> Bug Fixes. > >> >> >>>>>>>>> Provide your opinion on inclusion/exclusion of the above > >> points > >> >> or > >> >> >>>>>>> more > >> >> >>>>>>>> for the scope of the release. We can also, take some work > >> which > >> >> is > >> >> >>>>>>> already > >> >> >>>>>>>> done and review is pending for them (depending on the > >> >> availability > >> >> >>>>>>> of the > >> >> >>>>>>>> reviewers). > >> >> >>>>>>>>> > >> >> >>>>>>>>> Thanks, > >> >> >>>>>>>>> Abhishek > >> >> >>>>>>>> > >> >> >>>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>>>> > >> >> >>>>> > >> >> >>>> > >> >> >>> > >> >> >>> > >> >> > > >> >> > >> >> > >> > > >> > > > > >
