Hello All, How is it looking now?
I think we should move to blocker and critical patch mode. If required, we can take couple of days extra there. ~Abhishek > On Aug 19, 2016, at 11:31 AM, Abhishek Bafna <[email protected]> wrote: > > Sure Robert. We can use some more time. > > ~Abhishek >> On Aug 19, 2016, at 7:13 AM, Robert Kanter <[email protected]> wrote: >> >> Do you think we could wait one more week? I've been really hammered with >> some other work and there's more that I'd like to review here that I >> haven't been able to get to. >> >> thanks >> - Robert >> >> On Fri, Aug 12, 2016 at 2:30 AM, Abhishek Bafna <[email protected]> >> wrote: >> >>> Sure. I guess we can one extra week and complete some more work. >>> Thanks everyone for responding. >>> >>> ~Abhishek >>>> On Aug 12, 2016, at 2:46 PM, Purshotam Shah >>>> <[email protected]> >>> wrote: >>>> >>>> Yes. I also have some pending patches. >>>> >>>> On Friday, August 12, 2016 2:11 AM, Robert Kanter < >>> [email protected]> wrote: >>>> >>>> >>>> Ya, I could use some more time as well. I have some deadlines early next >>>> week, which has kept me from doing more reviews. >>>> >>>> thanks >>>> - Robert >>>> >>>> On Thu, Aug 11, 2016 at 11:04 AM, jaydeep vishwakarma < >>>> [email protected]> wrote: >>>> >>>>> Let see if we can wait for another week. So we can merge few more >>> important >>>>> jiras. Few of them I personally want to get reviewed and merged. >>>>> >>>>> Regards, >>>>> Jaydeep >>>>> >>>>> On Thu, Aug 11, 2016 at 4:09 PM, Abhishek Bafna <[email protected]> >>>>> wrote: >>>>> >>>>>> As previously discussed, we will be moving with whatever patches can be >>>>>> reviewed/merged till end of this week. >>>>>> >>>>>> But there are some which needs to be reviewed and merged, mainly >>>>> regarding >>>>>> the upgrades. Some of the patches require Java 1.7, so I think we >>> should >>>>>> review OOZIE-2036 and merged it. >>>>>> >>>>>> ~Abhishek >>>>>>> On Aug 4, 2016, at 12:37 AM, Rohini Palaniswamy < >>>>> [email protected]> >>>>>> wrote: >>>>>>> >>>>>>> Cleaned up that list and now no jira refers to "trunk". Patch >>>>> Available >>>>>>> jiras targeted for trunk now point to 4.4.0. >>>>>>> >>>>>>> https://issues.apache.org/jira/issues/?jql=project%20% >>>>>> 3D%20Oozie%20AND%20fixVersion%20%3D%204.3.0%20and%20status% >>>>>> 20%3D%20%22Patch%20Available%22%20ORDER%20BY%20status%20DESC >>>>>>> >>>>>>> There are 42 jiras to go. We can take a look mid next week to see if >>>>> any >>>>>>> pending ones needs to be pushed out. There are quite a few which are >>>>>>> unassigned. >>>>>>> >>>>>>> If there is any jira, someone would like to include in 4.3.0 please >>>>>> update >>>>>>> the jira. >>>>>>> >>>>>>> Regards, >>>>>>> Rohini >>>>>>> >>>>>>> On Wed, Aug 3, 2016 at 11:44 AM, Rohini Palaniswamy < >>>>>> [email protected] >>>>>>>> wrote: >>>>>>> >>>>>>>> Unset the open ones without sending emails. Will update the resolved, >>>>>>>> closed and Patch Available ones sending emails. >>>>>>>> >>>>>>>> Regards, >>>>>>>> Rohini >>>>>>>> >>>>>>>> On Wed, Aug 3, 2016 at 11:36 AM, Robert Kanter <[email protected] >>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Ya, let's unset those remaining open trunk ones. I agree, we need >>> to >>>>>> put >>>>>>>>> in some effort to clean out old JIRAs and to review Patch Available >>>>>> JIRAs. >>>>>>>>> >>>>>>>>> Also, I just created OOZIE-2625 to drop workflowgenerator. We can >>>>>> discuss >>>>>>>>> the issue more there, but I listed a few reasons on OOZIE-2625 about >>>>>> why. >>>>>>>>> >>>>>>>>> - Robert >>>>>>>>> >>>>>>>>> On Wed, Aug 3, 2016 at 9:21 AM, Rohini Palaniswamy < >>>>>>>>> [email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> https://issues.apache.org/jira/issues/?jql=project%20% >>>>>> 3D%20Oozie%20AND%20fixVersion%3D%20%22trunk%22%20and% >>>>>> 20status%20%3D%20open%20ORDER%20BY%20resolution%20ASC%2C% >>>>>> 20status%20DESC&startIndex=50 >>>>>>>>>> >>>>>>>>>> These are jiras open and marked trunk. Should we unset fix version >>>>> on >>>>>>>>> these >>>>>>>>>> or mark them for next release? Don't see majority of them being >>>>> worked >>>>>>>>> on >>>>>>>>>> even for next release. We need to do bug scrub and close many of >>>>> these >>>>>>>>> some >>>>>>>>>> time. >>>>>>>>>> >>>>>>>>>>> Instead >>>>>>>>>> of deleting "trunk", I think we're better off just renaming it to >>>>> the >>>>>>>>>> release number. >>>>>>>>>> There is some amount of relabeling either way. If we go with >>>>> unsetting >>>>>>>>> fix >>>>>>>>>> version, then we can as well delete trunk. Also, I did create a >>>>> 4.4.0 >>>>>>>>>> yesterday to move some of the patch available jiras from 4.3.0. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wed, Aug 3, 2016 at 8:59 AM, Rohini Palaniswamy < >>>>>>>>>> [email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> I think it would be better to send out email for the bulk change. >>>>> If >>>>>>>>>> there >>>>>>>>>>> are any issues or jiras wrongly marked someone can respond. We do >>>>>> that >>>>>>>>>> with >>>>>>>>>>> Pig and there are always couple of jiras that folks come back with >>>>>>>>>> feedback >>>>>>>>>>> on pulling back to release. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Aug 3, 2016 at 8:07 AM, Robert Kanter < >>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Abhishek, next time you do a bulk change, please remember to tell >>>>> it >>>>>>>>> not >>>>>>>>>>>> to >>>>>>>>>>>> send out an email for every change :) >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:28 PM, Robert Kanter < >>>>> [email protected] >>>>>>> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Rohini, that sounds good to me. Especially because we're on >>>>>>>>> "master" >>>>>>>>>>>>> branch but "trunk" version in JIRA and it gets a little >>>>> confusing. >>>>>>>>>>>> Instead >>>>>>>>>>>>> of deleting "trunk", I think we're better off just renaming it >>> to >>>>>>>>> the >>>>>>>>>>>>> release number. Otherwise, we'll have to re-label any current >>>>>>>>> "trunk" >>>>>>>>>>>>> fields. We can use 4.4.0 for now, but I imagine that the next >>>>>>>>> release >>>>>>>>>>>>> might end up being 5.0.0 (with Oozie on Yarn); we can always >>>>>>>>> rename it >>>>>>>>>>>>> later. >>>>>>>>>>>>> >>>>>>>>>>>>> I'm not sure if there's a specific rule here, but I suppose we >>>>>>>>> should >>>>>>>>>>>>> probably have a vote in the private list to make things official >>>>> as >>>>>>>>>>>> this is >>>>>>>>>>>>> a release-management-related change. Can you send an email? >>>>>>>>>>>>> >>>>>>>>>>>>> Abhishek, thanks for putting this all together. I think >>> Rohini's >>>>>>>>>>>>> suggestion of updating the JIRAs is the right next step. You're >>>>>>>>>>>> probably >>>>>>>>>>>>> best off using the bulk change feature instead of doing each >>> JIRA >>>>>>>>>>>>> individually. I should have time to do a bunch of reviews this >>>>>>>>> week, >>>>>>>>>>>> I've >>>>>>>>>>>>> actually already started. I should have some time to continue >>>>>>>>> doing >>>>>>>>>>>> more >>>>>>>>>>>>> reviews. As a general thing, we've been pretty slow about >>>>> getting >>>>>>>>>> these >>>>>>>>>>>>> done :( >>>>>>>>>>>>> >>>>>>>>>>>>> - Robert >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Aug 2, 2016 at 4:25 PM, Rohini Palaniswamy < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I also did mark some of the enhancement jiras that Satish was >>>>>>>>> working >>>>>>>>>>>> on >>>>>>>>>>>>>> and which might take time to review to 4.4.0 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>> Rohini >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Tue, Aug 2, 2016 at 3:01 PM, Rohini Palaniswamy < >>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> Abhishek, >>>>>>>>>>>>>>> Can you mark all jiras already committed to trunk as 4.3.0 >>>>>>>>> and >>>>>>>>>>>> those >>>>>>>>>>>>>>> planned in the scope document as 4.3.0? This will help filter >>>>> on >>>>>>>>>>>> those >>>>>>>>>>>>>>> and review instead of referring to a google doc. I can pick up >>>>>>>>>> 10-15 >>>>>>>>>>>>>>> jiras for review end of this week. We should try to get >>>>>>>>> majority of >>>>>>>>>>>>>> patches >>>>>>>>>>>>>>> in the next 10 days and branch by Aug 12th. You can then use >>>>> the >>>>>>>>>> 3rd >>>>>>>>>>>>>> week >>>>>>>>>>>>>>> for any pending critical issues and rolling the release >>>>>>>>> candidate. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Robert, >>>>>>>>>>>>>>> I am thinking, that we should get rid of the trunk version >>> and >>>>>>>>>> use >>>>>>>>>>>>>>> release numbers when committing from the beginning like all >>>>>>>>> other >>>>>>>>>>>>>> projects >>>>>>>>>>>>>>> do. Unnecessary step to change from trunk to release version >>>>> for >>>>>>>>>>>> every >>>>>>>>>>>>>>> release. If folks are in agreement, then will delete the trunk >>>>>>>>>>>> version >>>>>>>>>>>>>>> altogether. Does that require a mail to private? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>> Rohini >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Fri, Jul 29, 2016 at 3:38 AM, Abhishek Bafna < >>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> As we prepared a Oozie 4.3.0 scope document < >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G >>>>>> mFsuHQZmrT63Ow/edit#heading=h.w9ycxfry7sfu >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> and listed all the things which we wanted to include in the >>>>>>>>>> release. >>>>>>>>>>>>>> In the >>>>>>>>>>>>>>>> last few days, I spent some time and reviewed some of the >>>>>>>>> patches >>>>>>>>>>>> and >>>>>>>>>>>>>>>> provided my feedback. I will continue my efforts to review >>>>> more >>>>>>>>>>>>>> patches. >>>>>>>>>>>>>>>> These patches needs to be reviewed by committers and get >>>>>>>>>> committed. >>>>>>>>>>>> I >>>>>>>>>>>>>>>> request other contributors also, review some of the patches >>>>> and >>>>>>>>>>>> provide >>>>>>>>>>>>>>>> their feedback. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> I am thinking, may be, we can decide upon some fixed >>>>>>>>> (approximate) >>>>>>>>>>>>>>>> release date and start working accordingly. Because, If we >>>>> wait >>>>>>>>>> for >>>>>>>>>>>>>> all the >>>>>>>>>>>>>>>> listed items to be reviewed and committed, then it might take >>>>> a >>>>>>>>>> lot >>>>>>>>>>>>>> more >>>>>>>>>>>>>>>> time. I am thinking we should target third week of August to >>>>>>>>>>>> complete >>>>>>>>>>>>>> and >>>>>>>>>>>>>>>> release. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Feel free to provide your feedback and suggestions. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>> Abhishek >>>>>>>>>>>>>>>>> On Jul 21, 2016, at 4:39 PM, Abhishek Bafna < >>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Hi, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I looked into the available patches and prepared the list. I >>>>>>>>>> have >>>>>>>>>>>>>> put, >>>>>>>>>>>>>>>> scope content which we have discussed till now, into a word >>>>>>>>>>>> document. >>>>>>>>>>>>>> (For >>>>>>>>>>>>>>>> the better formatting and easy to read and comment). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> I have noted the additional Jira into the list. Added two >>>>>>>>> more >>>>>>>>>>>> Jira >>>>>>>>>>>>>>>> related to spark (Scala version and Spark 2.0 property). >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please take a look at the document (link below) and provide >>>>>>>>> your >>>>>>>>>>>>>>>> comments. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If this looks good, we can start the release process. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Scope Document: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> https://docs.google.com/document/d/1E1vGTul0hrWC4R8EgCRkO3ZmGNT5G >>>>>> mFsuHQZmrT63Ow/edit# >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>> Abhishek >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Jul 21, 2016, at 5:07 AM, Robert Kanter < >>>>>>>>>> [email protected] >>>>>>>>>>>>> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> That list looks good to me. The only one I'm not sure of >>> is >>>>>>>>>>>>>> OOZIE-2306 >>>>>>>>>>>>>>>>>> (Oozie Health Check). It's a new feature and it might make >>>>>>>>>>>> sense to >>>>>>>>>>>>>>>> let it >>>>>>>>>>>>>>>>>> "bake" a while first? I'm also not sure it's quite ready >>>>>>>>> yet. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'd also like to add a few others to the list: >>>>>>>>>>>>>>>>>> - OOZIE-2273 (MiniOozie does not work outside of Oozie) >>>>>>>>>>>>>>>>>> - The subtasks under OOZIE-2036 (Drop support for Java 1.6) >>>>>>>>>>>>>>>>>> --- OOZIE-2037 (Add TLSv1.1,TLSv1.2) >>>>>>>>>>>>>>>>>> --- OOZIE-1793 (Improve find bugs reporting for Oozie) >>>>>>>>>>>>>>>>>> --- OOZIE-2552 (Update ActiveMQ version for security and >>>>>>>>> other >>>>>>>>>>>>>> fixes) >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> There's a number of other Patch Available JIRAs not on your >>>>>>>>>> list >>>>>>>>>>>>>>>> (including >>>>>>>>>>>>>>>>>> the ones I mentioned above). While we don't need to >>> include >>>>>>>>>>>>>> everything >>>>>>>>>>>>>>>>>> here (and we probably shouldn't nor do we have the review >>>>>>>>>>>>>> bandwidth), >>>>>>>>>>>>>>>> did >>>>>>>>>>>>>>>>>> you consider each of them when creating the list? I >>>>>>>>> currently >>>>>>>>>>>> see >>>>>>>>>>>>>> 97 >>>>>>>>>>>>>>>>>> JIRAs: https://s.apache.org/oozie-patch-available >>>>>>>>>>>>>>>>>> I think we should include all Spark Action >>>>>>>>> fixes/improvements, >>>>>>>>>>>> given >>>>>>>>>>>>>>>> how >>>>>>>>>>>>>>>>>> popular it seems to be. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> thanks >>>>>>>>>>>>>>>>>> - Robert >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> On Wed, Jul 20, 2016 at 5:00 AM, Abhishek Bafna < >>>>>>>>>>>>>> [email protected]> >>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Below is the list of issues, picked from currently >>>>>>>>> available >>>>>>>>>>>>>> patches. >>>>>>>>>>>>>>>> I >>>>>>>>>>>>>>>>>>> could not decide up any 'open' issue, If there is >>> anything, >>>>>>>>>>>> please >>>>>>>>>>>>>>>> reply >>>>>>>>>>>>>>>>>>> here. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> We can 'drop/add' more if required based on the feedback. >>>>>>>>>>>> Important >>>>>>>>>>>>>>>> is, >>>>>>>>>>>>>>>>>>> how many of these we will be able to review and commit. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> I will starting looking into these issues and provide >>>>>>>>>> feedback. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> ------------------------------------------------------------ >>>>>> --------------- >>>>>>>>>>>>>>>>>>> OOZIE-1978 Forkjoin validation code is ridiculously >>>>>>>>> slow >>>>>>>>>> in >>>>>>>>>>>>>> some >>>>>>>>>>>>>>>> cases >>>>>>>>>>>>>>>>>>> OOZIE-2508 Documentation change for Coord action >>> rerun >>>>>>>>>>>>>>>> [OOZIE-1735] >>>>>>>>>>>>>>>>>>> OOZIE-2583 oozie throws EL Exception when reference >>>>>>>>>>>> variable >>>>>>>>>>>>>> name >>>>>>>>>>>>>>>>>>> containing dot >>>>>>>>>>>>>>>>>>> OOZIE-2582 Populating external child Ids for action >>>>>>>>>>>> failures >>>>>>>>>>>>>>>>>>> OOZIE-2517 Add support for startCreatedTime and >>>>>>>>>>>> endCreatedTime >>>>>>>>>>>>>>>>>>> filters for coord and bundles >>>>>>>>>>>>>>>>>>> OOZIE-2512 ShareLibservice returns incorrect path for >>>>>>>>> jar >>>>>>>>>>>>>>>>>>> OOZIE-2572 SLA DURATION miss not shown when job is >>>>>>>>>> running >>>>>>>>>>>> for >>>>>>>>>>>>>>>> longer >>>>>>>>>>>>>>>>>>> than expected time >>>>>>>>>>>>>>>>>>> OOZIE-2457 Oozie log parsing regex consume more than >>>>>>>>> 90% >>>>>>>>>>>> cpu >>>>>>>>>>>>>>>>>>> OOZIE-2574 Oozie to support replication-enabled mysql >>>>>>>>>> urls >>>>>>>>>>>>>>>>>>> OOZIE-2506 Add logs into RecoverService for logging >>>>>>>>>>>>>> information >>>>>>>>>>>>>>>> about >>>>>>>>>>>>>>>>>>> queued commnads >>>>>>>>>>>>>>>>>>> OOZIE-2573 dataset url which contains spaces can not >>>>>>>>> be >>>>>>>>>>>>>> handled >>>>>>>>>>>>>>>>>>> rightly >>>>>>>>>>>>>>>>>>> OOZIE-2569 Adding yarn-site, core-site, hdfs-site and >>>>>>>>>>>>>> mapred-site >>>>>>>>>>>>>>>>>>> into spark launcher >>>>>>>>>>>>>>>>>>> OOZIE-2491 oozie acl cannot specify group,it does`t >>>>>>>>> work >>>>>>>>>>>>>>>>>>> OOZIE-2539 Incorrect property key is used for 'hive >>>>>>>>> log4j >>>>>>>>>>>>>>>>>>> configuration file for execution mode' >>>>>>>>>>>>>>>>>>> OOZIE-2555 Oozie SSL enable setup does not return >>> port >>>>>>>>>> for >>>>>>>>>>>>>> admin >>>>>>>>>>>>>>>>>>> -servers >>>>>>>>>>>>>>>>>>> OOZIE-2515 Duplicate information for "Changing >>>>>>>>>>>>>> endtime/pausetime >>>>>>>>>>>>>>>> of a >>>>>>>>>>>>>>>>>>> Bundle Job" in CommandLineTool wiki >>>>>>>>>>>>>>>>>>> OOZIE-2498 Oozie CallerId configuration for >>> downstream >>>>>>>>>>>>>> components >>>>>>>>>>>>>>>>>>> OOZIE-2516 Update web service documentation for jobs >>>>>>>>> API >>>>>>>>>>>>>>>>>>> OOZIE-2500 -DtestJarSimple option mentioned in >>>>>>>>> minioozie >>>>>>>>>>>> doc >>>>>>>>>>>>>>>> does not >>>>>>>>>>>>>>>>>>> work >>>>>>>>>>>>>>>>>>> OOZIE-2513 log.scan.duration should not be used for >>>>>>>>> error >>>>>>>>>>>> and >>>>>>>>>>>>>>>> audit >>>>>>>>>>>>>>>>>>> logs >>>>>>>>>>>>>>>>>>> OOZIE-2461 Workflow, Coordinator and Bundle job >>>>>>>>> querying >>>>>>>>>>>>>> should >>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>>> last modified filter >>>>>>>>>>>>>>>>>>> OOZIE-2306 Oozie health check >>>>>>>>>>>>>>>>>>> OOZIE-2518 Oozie is creating many files and >>>>>>>>> directories >>>>>>>>>> in >>>>>>>>>>>>>> /tmp >>>>>>>>>>>>>>>> per >>>>>>>>>>>>>>>>>>> day >>>>>>>>>>>>>>>>>>> OOZIE-2409 hive2 action with hive 1.2.1 failed >>>>>>>>>>>>>>>>>>> OOZIE-2473 Connection pool for SMTP connection >>>>>>>>>>>>>>>>>>> OOZIE-2440 Exponential re-try policy for workflow >>>>>>>>> action >>>>>>>>>>>>>>>>>>> OOZIE-2458 'oozie-setup.sh sharelib create' should >>>>>>>>> ensure >>>>>>>>>>>>>>>> uploaded >>>>>>>>>>>>>>>>>>> jars are world readable >>>>>>>>>>>>>>>>>>> OOZIE-2390 Rerun with failed option removing >>> completed >>>>>>>>>>>> output >>>>>>>>>>>>>>>> data >>>>>>>>>>>>>>>>>>> OOZIE-1810 Workflow cannot get into Failed state when >>>>>>>>>> kill >>>>>>>>>>>>>>>> control >>>>>>>>>>>>>>>>>>> node cannot resolve variable in message >>>>>>>>>>>>>>>>>>> OOZIE-2244 Oozie should mask passwords in the logs >>>>>>>>> when >>>>>>>>>>>>>> logging >>>>>>>>>>>>>>>>>>> command arguments >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>> >>>>>>>>> ------------------------------------------------------------ >>>>>> --------------- >>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>> Abhishek >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On Jul 18, 2016, at 6:35 PM, Abhishek Bafna < >>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> I looked into the commits (approx. 160), all looked good >>>>>>>>> to >>>>>>>>>> me >>>>>>>>>>>> and >>>>>>>>>>>>>>>>>>> should be included into the release. If there are things, >>>>>>>>>> which >>>>>>>>>>>>>> needs >>>>>>>>>>>>>>>> to be >>>>>>>>>>>>>>>>>>> excluded, provide your comments. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> We still need to decide on the patch which are in >>> progress >>>>>>>>>>>> status. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> On the upgrade side: >>>>>>>>>>>>>>>>>>>> Java version 1.6 to 1.7 >>>>>>>>>>>>>>>>>>>> OpenJPA version 2.2.2 to 2.4.1 >>>>>>>>>>>>>>>>>>>> Curator version 2.5.0 to 2.11.0 >>>>>>>>>>>>>>>>>>>> Pig version 0.12.1 to 0.16.0 >>>>>>>>>>>>>>>>>>>> Hive version 0.13.1 to 1.2.0 >>>>>>>>>>>>>>>>>>>> HttpClient version 4.3 to 4.3.6 >>>>>>>>>>>>>>>>>>>> commons-httpclient version 3.1 to 4.3.6 (discontinued) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Please provide your feedback on the above upgrades. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Once we decide on the scope of the release. We will start >>>>>>>>>>>> working >>>>>>>>>>>>>> on >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> completing/testing the release. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>> Abhishek >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:40 AM, Abhishek Bafna < >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Okay. We will make the Java version to 1.7. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> There are upgrades we can do like OpenJPA, Curator, >>>>>>>>>>>> HttpClient, >>>>>>>>>>>>>> Pig >>>>>>>>>>>>>>>> etc. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> HttpClient, Pig (0.16) version upgrade we have done >>>>>>>>>>>> internally as >>>>>>>>>>>>>>>> well. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> As per the comments on the OOZIE-2231 < >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231>, >>> curator >>>>>>>>>>>> version >>>>>>>>>>>>>>>> also >>>>>>>>>>>>>>>>>>> can be upgraded to 2.11. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>> Abhishek >>>>>>>>>>>>>>>>>>>>>> On Jul 16, 2016, at 2:06 AM, Robert Kanter < >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> We've been using a version of Hive/HCat based on 1.1.0 >>>>>>>>> for >>>>>>>>>>>> over >>>>>>>>>>>>>> a >>>>>>>>>>>>>>>> year >>>>>>>>>>>>>>>>>>> now >>>>>>>>>>>>>>>>>>>>>> with no problems. We did have to comment out a few >>>>>>>>> lines >>>>>>>>>> in >>>>>>>>>>>>>>>>>>> MiniHCatServer >>>>>>>>>>>>>>>>>>>>>> and MiniHS2 that were no longer needed: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> diff --git >>>>>>>>>>>>>>>>>>> >>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>>>>>>>>>>>>>>>>>>>>> index 8699ff8..d694977 100644 >>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>>>>>>>>>>>>>>>>>>>>> +++ >>>>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/MiniHCatServer.java >>>>>>>>>>>>>>>>>>>>>> @@ -150,9 +150,10 @@ public class MiniHCatServer { >>>>>>>>>>>>>>>>>>>>>> public static void resetDefaultDBCreation() throws >>>>>>>>>>>> Exception { >>>>>>>>>>>>>>>>>>>>>> // Need to do this, else default db will not be >>>>>>>>>> created >>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>>>> local >>>>>>>>>>>>>>>>>>>>>> metastores. >>>>>>>>>>>>>>>>>>>>>> // TestHiveMain will fail with >>>>>>>>>>>>>>>>>>>>>> InvalidObjectException(message:There is no database >>>>>>>>> named >>>>>>>>>>>>>> default) >>>>>>>>>>>>>>>>>>>>>> - Field declaredField = >>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); >>>>>>>>>>>>>>>>>>>>>> - declaredField.setAccessible(true); >>>>>>>>>>>>>>>>>>>>>> - declaredField.set(null, false); >>>>>>>>>>>>>>>>>>>>>> + // CLOUDERA-BUILD: Don't need this for Hive >>>>>>>>> 0.14 >>>>>>>>>> and >>>>>>>>>>>>>> later >>>>>>>>>>>>>>>>>>>>>> + //Field declaredField = >>>>>>>>>>>>>>>>>>>>>> HMSHandler.class.getDeclaredField("createDefaultDB"); >>>>>>>>>>>>>>>>>>>>>> + //declaredField.setAccessible(true); >>>>>>>>>>>>>>>>>>>>>> + //declaredField.set(null, false); >>>>>>>>>>>>>>>>>>>>>> } >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> public static void resetHiveConfStaticVariables() >>>>>>>>> throws >>>>>>>>>>>>>>>> Exception >>>>>>>>>>>>>>>>>>> { >>>>>>>>>>>>>>>>>>>>>> diff --git >>>>>>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>>>>>>>>>>>>>>>>>>>>> index c9dfd56..505dc55 100644 >>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>> a/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>>>>>>>>>>>>>>>>>>>>> +++ >>>>>>>>>>>> b/core/src/test/java/org/apache/oozie/test/hive/MiniHS2.java >>>>>>>>>>>>>>>>>>>>>> @@ -84,7 +84,8 @@ public class MiniHS2 extends >>>>>>>>>>>>>> AbstractHiveService >>>>>>>>>>>>>>>> { >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>> hiveConf.setVar(ConfVars.HIVE_SERVER2_THRIFT_BIND_HOST, >>>>>>>>>>>>>>>> getHost()); >>>>>>>>>>>>>>>>>>>>>> hiveConf.setIntVar(ConfVars. >>> HIVE_SERVER2_THRIFT_PORT, >>>>>>>>>>>>>>>>>>> getBinaryPort()); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>> hiveConf.setIntVar(ConfVars.HIVE_SERVER2_THRIFT_HTTP_PORT, >>>>>>>>>>>>>>>>>>>>>> getHttpPort()); >>>>>>>>>>>>>>>>>>>>>> - HiveMetaStore.HMSHandler.resetDefaultDBFlag(); >>>>>>>>>>>>>>>>>>>>>> + // CLOUDERA-BUILD: Don't need this for Hive 0.14 >>>>>>>>> and >>>>>>>>>>>> later >>>>>>>>>>>>>>>>>>>>>> + //HiveMetaStore.HMSHandler.resetDefaultDBFlag(); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Path scratchDir = new Path(baseDfsDir, "scratch"); >>>>>>>>>>>>>>>>>>>>>> fs.mkdirs(scratchDir); >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 1:29 PM, Venkat Ranganathan < >>>>>>>>>>>>>>>>>>>>>> [email protected] <mailto: >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Let us explore changing the default hive version also >>>>>>>>> to >>>>>>>>>>>> 1.x. >>>>>>>>>>>>>>>> 0.13 >>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>> old >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Venkat >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On 7/15/16, 11:31 AM, "Robert Kanter" < >>>>>>>>>> [email protected] >>>>>>>>>>>>>>>> <mailto: >>>>>>>>>>>>>>>>>>> [email protected]>> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I don't think we have any guidelines defined anywhere >>>>>>>>>> about >>>>>>>>>>>>>> when >>>>>>>>>>>>>>>>>>> it's >>>>>>>>>>>>>>>>>>>>>>> okay >>>>>>>>>>>>>>>>>>>>>>> to change the minimum Java version, but perhaps we >>>>>>>>> need >>>>>>>>>> to >>>>>>>>>>>>>> wait >>>>>>>>>>>>>>>>>>> until >>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> next major release (5.0)? That said, Hadoop did this >>>>>>>>> in >>>>>>>>>> a >>>>>>>>>>>>>> minor >>>>>>>>>>>>>>>>>>>>>>> release >>>>>>>>>>>>>>>>>>>>>>> and Java 6 was EoL in 2013. So I'm fine with doing >>>>>>>>> it in >>>>>>>>>>>> 4.3 >>>>>>>>>>>>>> if >>>>>>>>>>>>>>>>>>>>>>> nobody has >>>>>>>>>>>>>>>>>>>>>>> an objections; we should just make sure to call that >>>>>>>>> out >>>>>>>>>> in >>>>>>>>>>>>>> our >>>>>>>>>>>>>>>>>>> release >>>>>>>>>>>>>>>>>>>>>>> notes. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Otherwise, nothing comes to mind for anything we >>>>>>>>> should >>>>>>>>>>>> leave >>>>>>>>>>>>>> out >>>>>>>>>>>>>>>>>>> other >>>>>>>>>>>>>>>>>>>>>>> than the Oozie on Yarn stuff, which is not currently >>>>>>>>> in >>>>>>>>>>>> trunk >>>>>>>>>>>>>>>>>>> anyway. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> I know there's a number of Spark Action JIRAs; we >>>>>>>>> should >>>>>>>>>>>> make >>>>>>>>>>>>>>>> sure >>>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>>>>>>> include all of these and any pending ones. >>>>>>>>>>>>>>>>>>>>>>> @Rohini, @Puru, @Satish, anything you think needs more >>>>>>>>>>>> time to >>>>>>>>>>>>>>>> bake >>>>>>>>>>>>>>>>>>>>>>> that we >>>>>>>>>>>>>>>>>>>>>>> should hold off on including in the release? >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> thanks >>>>>>>>>>>>>>>>>>>>>>> - Robert >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Fri, Jul 15, 2016 at 8:35 AM, Abhishek Bafna < >>>>>>>>>>>>>>>>>>> [email protected] <mailto:[email protected]>> >>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Any suggestions or comments on the scope for the >>>>>>>>> release? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>> Abhishek >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Jul 13, 2016, at 3:07 AM, Abhishek Bafna < >>>>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>>>>>>> <mailto:[email protected]>> >>>>>>>>>>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Hi All, >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Oozie is starting the release process for Oozie >>>>>>>>> 4.3.0. >>>>>>>>>>>> This >>>>>>>>>>>>>>>> thread >>>>>>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>> for discussion about the scope of the release. Please >>>>>>>>>>>> provide >>>>>>>>>>>>>>>> your >>>>>>>>>>>>>>>>>>>>>>>> suggestion about what should be included (or any >>>>>>>>>>>> exclusions) >>>>>>>>>>>>>> in >>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>> release. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Currently, we have around 158 patches submitted to >>>>>>>>>> trunk. >>>>>>>>>>>>>> Based >>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>> scope, we will include (exclude) more if required. >>>>>>>>>>>>>>>>>>>>>>>>> Version Upgrade >>>>>>>>>>>>>>>>>>>>>>>>> Java 1.7 (OOZIE-2036 < >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036 < >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2036>>) >>>>>>>>>>>>>>>>>>>>>>>>> OpenJPA (OOZIE-2488 < >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488 < >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2488>>), >>>>>>>>>>>>>>>>>>>>>>>>> Curator (OOZIE-2231 < >>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231 < >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2231> >>>>>>>>>>>>>>>>>>>>>>>>> ), >>>>>>>>>>>>>>>>>>>>>>>>> HttpClient (OOZIE-2538 < >>>>>>>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538 < >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2538>>), >>>>>>>>>>>>>>>>>>>>>>>>> Pig (OOZIE-2602 < >>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602 < >>>>>>>>>>>>>>>>>>> https://issues.apache.org/jira/browse/OOZIE-2602> >>>>>>>>>>>>>>>>>>>>>>>> ). >>>>>>>>>>>>>>>>>>>>>>>>> New Features/Improvements: Add default JT and NN, >>>>>>>>>> Database >>>>>>>>>>>>>>>> schema >>>>>>>>>>>>>>>>>>>>>>>> service check, instrumental metrics, Tomcat upgrade >>>>>>>>> etc. >>>>>>>>>>>>>>>>>>>>>>>>> Bug Fixes. >>>>>>>>>>>>>>>>>>>>>>>>> Provide your opinion on inclusion/exclusion of the >>>>>>>>> above >>>>>>>>>>>>>> points >>>>>>>>>>>>>>>> or >>>>>>>>>>>>>>>>>>>>>>> more >>>>>>>>>>>>>>>>>>>>>>>> for the scope of the release. We can also, take some >>>>>>>>> work >>>>>>>>>>>>>> which >>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>> already >>>>>>>>>>>>>>>>>>>>>>>> done and review is pending for them (depending on the >>>>>>>>>>>>>>>> availability >>>>>>>>>>>>>>>>>>>>>>> of the >>>>>>>>>>>>>>>>>>>>>>>> reviewers). >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>>>>>>>>>>>>> Abhishek >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >
