Couple of more JIRAs need to be back ported for 2.7.4 release. These will solve RM HA unstability issues. https://issues.apache.org/jira/browse/YARN-5333 https://issues.apache.org/jira/browse/YARN-5988 https://issues.apache.org/jira/browse/YARN-6304
I will raise a JIRAs to back port it. @Akira , could you help to add these JIRAs into wiki? Thanks & Regards Rohith Sharma K S On 29 May 2017 at 12:19, Akira Ajisaka <aajis...@apache.org> wrote: > Created a page for 2.7.4 release. > https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.7.4 > > If you want to edit this wiki, please ping me. > > Regards, > Akira > > > On 2017/05/23 4:42, Brahma Reddy Battula wrote: > >> Hi Konstantin Shvachko >> >> >> how about creating a wiki page for 2.7.4 release status like 2.8 and >> trunk in following link.?? >> >> >> https://cwiki.apache.org/confluence/display/HADOOP >> >> >> ________________________________ >> From: Konstantin Shvachko <shv.had...@gmail.com> >> Sent: Saturday, May 13, 2017 3:58 AM >> To: Akira Ajisaka >> Cc: Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org; >> yarn-dev@hadoop.apache.org >> Subject: Re: About 2.7.4 Release >> >> Latest update on the links and filters. Here is the correct link for the >> filter: >> https://issues.apache.org/jira/secure/IssueNavigator.jspa? >> requestId=12340814 >> >> Also updated: https://s.apache.org/Dzg4 >> >> Had to do some Jira debugging. Sorry for confusion. >> >> Thanks, >> --Konstantin >> >> On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko < >> shv.had...@gmail.com> >> wrote: >> >> Hey Akira, >>> >>> I didn't have private filters. Most probably Jira caches something. >>> Your filter is in the right direction, but for some reason it lists only >>> 22 issues, while mine has 29. >>> It misses e.g. YARN-5543 <https://issues.apache.org/jir >>> a/browse/YARN-5543> >>> . >>> >>> Anyways, I created a Jira filter now "Hadoop 2.7.4 release blockers", >>> shared it with "everybody", and updated my link to point to that filter. >>> So >>> you can use any of the three methods below to get the correct list: >>> 1. Go to https://s.apache.org/Dzg4 >>> 2. Go to the filter via >>> https://issues.apache.org/jira/issues?filter=12340814 >>> or by finding "Hadoop 2.7.4 release blockers" filter in the jira >>> 3. On Advanced issues search page paste this: >>> project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels = release-blocker >>> AND "Target Version/s" = 2.7.4 >>> >>> Hope this solves the confusion for which issues are included. >>> Please LMK if it doesn't, as it is important. >>> >>> Thanks, >>> --Konstantin >>> >>> On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka <aajis...@apache.org> >>> wrote: >>> >>> Hi Konstantin, >>>> >>>> Thank you for volunteering as release manager! >>>> >>>> Actually the original link works fine: https://s.apache.org/Dzg4 >>>>> >>>> I couldn't see the link. Maybe is it private filter? >>>> >>>> Here is a link I generated: https://s.apache.org/ehKy >>>> This filter includes resolved issue and excludes fixversion == 2.7.4 >>>> >>>> Thanks and Regards, >>>> Akira >>>> >>>> On 2017/05/08 19:20, Konstantin Shvachko wrote: >>>> >>>> Hi Brahma Reddy Battula, >>>>> >>>>> Actually the original link works fine: https://s.apache.org/Dzg4 >>>>> Your link excludes closed and resolved issues, which needs backporting, >>>>> and >>>>> which we cannot reopen, as discussed in this thread earlier. >>>>> >>>>> Looked through the issues you proposed: >>>>> >>>>> HDFS-9311 <https://issues.apache.org/jira/browse/HDFS-9311> >>>>> Seems like a new feature. It helps failover to standby node when >>>>> primary >>>>> is >>>>> under heavy load, but it introduces new APIs, addresses, config >>>>> parameters. >>>>> And needs at least one follow up jira. >>>>> Looks like a backward compatible change, though. >>>>> Did you have a chance to run it in production? >>>>> >>>>> +1 on >>>>> HDFS-10987 <https://issues.apache.org/jira/browse/HDFS-10987> >>>>> >>>> [HDFS-10987] Make Decommission less expensive when lot of ...< >> https://issues.apache.org/jira/browse/HDFS-10987> >> issues.apache.org >> When user want to decommission a node which having 50M blocks ,it could >> hold the namesystem lock for long time.We've seen it is taking 36 sec. As >> we knew during this ... >> >> >> >> HDFS-9902 <https://issues.apache.org/jira/browse/HDFS-9902> >>>>> >>>> [HDFS-9902] Support different values of dfs.datanode.du ...< >> https://issues.apache.org/jira/browse/HDFS-9902> >> issues.apache.org >> Now Hadoop support different storage type for DISK, SSD, ARCHIVE and >> RAM_DISK, but they share one configuration dfs.datanode.du.reserved. The >> DISK size may be several ... >> >> >> >> HDFS-8312 <https://issues.apache.org/jira/browse/HDFS-8312> >>>>> >>>> Trash does not descent into child directories to check for ...< >> https://issues.apache.org/jira/browse/HDFS-8312> >> issues.apache.org >> HDFS trash does not descent into child directory to check if user has >> permission to delete files. For example: Run the following command to >> initialize directory ... >> >> >> >> HADOOP-14100 <https://issues.apache.org/jira/browse/HADOOP-14100> >>>>> >>>> Upgrade Jsch jar to latest version to fix vulnerability in ...< >> https://issues.apache.org/jira/browse/HADOOP-14100> >> issues.apache.org >> Recently there was on vulnerability reported on jsch library. Its fixed >> in latest 0.1.54 version before CVE was made public. >> https://cve.mitre.org/cgi-bin/cvename.cgi ... >> >> >> >> >>>>> Added them to 2.7.4 release. You should see them via the above link >>>>> now. >>>>> Would be good if you could attach backport patches for some of them? >>>>> >>>>> Appreciate your help, >>>>> --Konstantin >>>>> >>>>> On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy Battula < >>>>> brahmareddy.batt...@huawei.com> wrote: >>>>> >>>>> >>>>> Looks following link is not correct.. >>>>>> >>>>>> https://s.apache.org/Dzg4 >>>>>> >>>>>> It should be like following..? >>>>>> >>>>>> https://s.apache.org/wi3U >>>>>> >>>>>> >>>>>> Apart from Konstantin mentioned,Following also good to go..? let me >>>>>> know >>>>>> your thoughts on this. >>>>>> >>>>>> For Large Cluster: >>>>>> ============= >>>>>> >>>>>> https://issues.apache.org/jira/browse/HDFS-9311===Life Line Protocol >>>>>> https://issues.apache.org/jira/browse/HDFS-10987===Deecommission >>>>>> Expensive when lot's of blocks are present >>>>>> >>>>>> https://issues.apache.org/jira/browse/HDFS-9902=== >>>>>> "dfs.datanode.du.reserved" per Storage Type >>>>>> >>>>>> For Security: >>>>>> ========= >>>>>> https://issues.apache.org/jira/browse/HDFS-8312===Trash does not >>>>>> descent >>>>>> into child directories to check for permission >>>>>> https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade Jsch jar >>>>>> to >>>>>> latest version to fix vulnerability in old versions >>>>>> >>>>>> >>>>>> >>>>>> Regards >>>>>> Brahma Reddy Battula >>>>>> >>>>>> -----Original Message----- >>>>>> From: Erik Krogen [mailto:ekro...@linkedin.com.INVALID] >>>>>> Sent: 06 May 2017 02:40 >>>>>> To: Konstantin Shvachko >>>>>> Cc: Zhe Zhang; Hadoop Common; Hdfs-dev; >>>>>> mapreduce-...@hadoop.apache.org >>>>>> ; >>>>>> yarn-dev@hadoop.apache.org >>>>>> Subject: Re: About 2.7.4 Release >>>>>> >>>>>> List LGTM Konstantin! >>>>>> >>>>>> Let's say that we will only create a new tracking JIRA for patches >>>>>> which >>>>>> do not backport cleanly, to avoid having too many lying around. >>>>>> Otherwise >>>>>> we can directly attach to old ticket. If a clean backport does happen >>>>>> to >>>>>> break a test the nightly build will help us catch it. >>>>>> >>>>>> Erik >>>>>> >>>>>> On Thu, May 4, 2017 at 7:21 PM, Konstantin Shvachko < >>>>>> shv.had...@gmail.com> >>>>>> wrote: >>>>>> >>>>>> Great Zhe. Let's monitor the build. >>>>>> >>>>>>> >>>>>>> I marked all jiras I knew of for inclusion into 2.7.4 as I described >>>>>>> before. >>>>>>> Target Version/s: 2.7.4 >>>>>>> Label: release-blocker >>>>>>> >>>>>>> Here is the link to the list: https://s.apache.org/Dzg4 Please let >>>>>>> me >>>>>>> know if I missed anything. >>>>>>> And feel free to pick up any. Most of backports are pretty >>>>>>> straightforward, but not all. >>>>>>> >>>>>>> We can create tracking jiras for backporting if you need to run >>>>>>> Jenkins on the patch (and since Allen does not allow reopening them). >>>>>>> But I think the final patch should be attached to the original jira. >>>>>>> Otherwise history will be hard to follow. >>>>>>> >>>>>>> Thanks, >>>>>>> --Konstantin >>>>>>> >>>>>>> On Wed, May 3, 2017 at 4:53 PM, Zhe Zhang <z...@apache.org> wrote: >>>>>>> >>>>>>> Thanks for volunteering as RM Konstantin! The plan LGTM. >>>>>>> >>>>>>>> >>>>>>>> I've created a nightly Jenkins job for branch-2.7 (unit tests): >>>>>>>> https://builds.apache.org/job/Hadoop-branch2.7-nightly/ >>>>>>>> >>>>>>>> On Wed, May 3, 2017 at 12:42 AM Konstantin Shvachko < >>>>>>>> >>>>>>>> shv.had...@gmail.com> >>>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> Hey guys, >>>>>>>> >>>>>>>>> >>>>>>>>> I and a few of my colleagues would like to help here and move 2.7.4 >>>>>>>>> release forward. A few points in this regard. >>>>>>>>> >>>>>>>>> 1. Reading through this thread since March 1 I see that Vinod >>>>>>>>> hinted on managing the release. Vinod, if you still want the job / >>>>>>>>> have bandwidth will be happy to work with you. >>>>>>>>> Otherwise I am glad to volunteer as the release manager. >>>>>>>>> >>>>>>>>> 2. In addition to current blockers and criticals, I would like to >>>>>>>>> >>>>>>>>> propose >>>>>>>> >>>>>>> >>>>>>> a >>>>>>>> >>>>>>>>> few issues to be included in the release, see the list below. Those >>>>>>>>> are mostly bug fixes and optimizations, which we already have in >>>>>>>>> our >>>>>>>>> >>>>>>>>> internal >>>>>>>> >>>>>>> >>>>>>> branch and run in production. Plus one minor feature "node >>>>>>>> >>>>>>>>> labeling", which we found very handy, when you have heterogeneous >>>>>>>>> environments and mixed workloads, like MR and Spark. >>>>>>>>> >>>>>>>>> 3. For marking issues for the release I propose to >>>>>>>>> - set the target version to 2.7.4, and >>>>>>>>> - add a new label "release-blocker" >>>>>>>>> That way we will know issues targeted for the release without >>>>>>>>> reopening them for backports. >>>>>>>>> >>>>>>>>> 4. I see quite a few people are interested in the release. With all >>>>>>>>> the help I think we can target to release by the end of May. >>>>>>>>> >>>>>>>>> Other things include fixing CHANGES.txt and fixing Jenkins build >>>>>>>>> for >>>>>>>>> >>>>>>>>> 2.7.4 >>>>>>>> >>>>>>> >>>>>>> branch. >>>>>>>> >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> --Konstantin >>>>>>>>> >>>>>>>>> ========== List of issue for 2.7.4 =========== >>>>>>>>> ------ Backports >>>>>>>>> HADOOP-12975 <https://issues.apache.org/jira/browse/HADOOP-12975>. >>>>>>>>> Add >>>>>>>>> >>>>>>>>> du >>>>>>>> >>>>>>> >>>>>>> jitters >>>>>>>> >>>>>>>>> HDFS-9710 <https://issues.apache.org/jira/browse/HDFS-9710>. IBR >>>>>>>>> >>>>>>>>> batching >>>>>>>> >>>>>>> >>>>>>> HDFS-10715 <https://issues.apache.org/jira/browse/HDFS-10715>. NPE >>>>>>>> >>>>>>>>> when applying AvailableSpaceBlockPlacementPolicy >>>>>>>>> HDFS-2538 <https://issues.apache.org/jira/browse/HDFS-2538>. fsck >>>>>>>>> >>>>>>>>> removal >>>>>>>> >>>>>>> >>>>>>> of dot printing >>>>>>>> >>>>>>>>> HDFS-8131 <https://issues.apache.org/jira/browse/HDFS-8131>. >>>>>>>>> space-balanced >>>>>>>>> policy for balancer >>>>>>>>> HDFS-8549 <https://issues.apache.org/jira/browse/HDFS-8549>. abort >>>>>>>>> balancer if upgrade in progress >>>>>>>>> HDFS-9412 <https://issues.apache.org/jira/browse/HDFS-9412>. skip >>>>>>>>> small blocks in getBlocks >>>>>>>>> >>>>>>>>> YARN-1471 <https://issues.apache.org/jira/browse/YARN-1471>. SLS >>>>>>>>> simulator >>>>>>>>> YARN-4302 <https://issues.apache.org/jira/browse/YARN-4302>. SLS >>>>>>>>> YARN-4367 <https://issues.apache.org/jira/browse/YARN-4367>. SLS >>>>>>>>> YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612>. SLS >>>>>>>>> >>>>>>>>> ----- Node labeling >>>>>>>>> MAPREDUCE-6304 >>>>>>>>> <https://issues.apache.org/jira/browse/MAPREDUCE-6304> >>>>>>>>> YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943> >>>>>>>>> YARN-4109 <https://issues.apache.org/jira/browse/YARN-4109> >>>>>>>>> YARN-4140 <https://issues.apache.org/jira/browse/YARN-4140> >>>>>>>>> YARN-4250 <https://issues.apache.org/jira/browse/YARN-4250> >>>>>>>>> YARN-4925 <https://issues.apache.org/jira/browse/YARN-4925> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> >>>>>>>> Zhe Zhang >>>>>>>> Apache Hadoop Committer >>>>>>>> http://zhe-thoughts.github.io/about/ | @oldcap >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>> >> > --------------------------------------------------------------------- > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > >
