Hi Sanjay,
Actually I was not aware of that work… This seems to be a better way of
achieving some of the same things we do externally to the DN process. I will
look into reimplementing some parts on top of this; seems it should just
require some very small extensions to DataNodeCluster. Thank you very much for
the pointer!
Erik
On 7/21/17, 11:01 AM, "sanjay Radia" <sanjayo...@gmail.com> wrote:
Erik
Great stuff.
BTW did you build on top of the “simulated data nodes” in HDFS which has a
way to storing only the length of data (but not real data)? That work allowed
supplementing with a matching editsLog for the NN. Your approach of using a
real image has the advantage of being able to replay traces from audit logs.
(Ref
https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DataNodeCluster.java)
thanks
sanjay
> On Jul 20, 2017, at 10:42 AM, Erik Krogen <ekro...@linkedin.com.INVALID>
wrote:
>
> forking off of the 2.7.4 release thread to answer this question about
> Dynamometer
>
> Dynamometer is a tool developed at LinkedIn for scale testing HDFS,
> specifically the NameNode. We have been using it for some time now and
have
> recently been making some enhancements to ease of use and reproducibility.
> We hope to post a blog post sometime in the not-too-distant future, and
> also to open source it. I can provide some details here given that we have
> been leveraging it as part of our 2.7.4 release / upgrade process (in
> addition to previous upgrades).
>
> The basic idea is to get full-scale black-box testing of the HDFS NN while
> using significantly less (~10%) hardware than a real cluster of that size
> would require. We use real NN images from our at-scale clusters paired
with
> some logic to fake out DNs into thinking they are storing data when they
> are not, allowing us to stuff more DNs onto each machine. Since we use a
> real image, we can replay real traces (collected from audit logs) to
> compare actual production performance vs. performance on this simulated
> cluster (with additional tuning, different version, etc.). We leverage
YARN
> to manage setting up this cluster and to replay the traces.
>
> Happy to answer questions.
>
> Erik
>
> On Wed, Jul 19, 2017 at 5:05 PM, Konstantin Shvachko
<shv.had...@gmail.com>
> wrote:
>
>> Hi Tianyi,
>>
>> Glad you are interested in Dynamometer. Erik (CC-ed) is actively working
>> on this project right now, I'll let him elaborate.
>> Erik, you should probably respond on Apache dev list, as I think it could
>> be interesting for other people as well, asince we planned to open source
>> it. You can fork the "About 2.7.4 Release" thread with a new subject and
>> give some details about Dynamometer there.
>>
>> Thanks,
>> --Konstantin
>>
>> On Wed, Jul 19, 2017 at 1:40 AM, 何天一 <hetia...@bytedance.com> wrote:
>>
>>> Hi, Shavachko.
>>>
>>> You mentioned an internal tool called Dynamometer to test NameNode
>>> performance earlier in the 2.7.4 release thread.
>>> I wonder if you could share some ideas behind the tool. Or is there a
>>> plan to bring Dynamometer to open source community?
>>>
>>> Thanks.
>>>
>>> BR,
>>> Tianyi
>>>
>>> On Fri, Jul 14, 2017 at 8:45 AM Konstantin Shvachko
<shv.had...@gmail.com>
>>> wrote:
>>>
>>>> Hi everybody.
>>>>
>>>> We have been doing some internal testing of Hadoop 2.7.4. The testing
is
>>>> going well.
>>>> Did not find any major issues on our workloads.
>>>> Used an internal tool called Dynamometer to check NameNode performance
on
>>>> real cluster traces. Good.
>>>> Overall test cluster performance looks good.
>>>> Some more testing is still going on.
>>>>
>>>> I plan to build an RC next week. If there are no objection.
>>>>
>>>> Thanks,
>>>> --Konst
>>>>
>>>> On Thu, Jun 15, 2017 at 4:42 PM, Konstantin Shvachko <
>>>> shv.had...@gmail.com>
>>>> wrote:
>>>>
>>>>> Hey guys.
>>>>>
>>>>> An update on 2.7.4 progress.
>>>>> We are down to 4 blockers. There is some work remaining on those.
>>>>> https://issues.apache.org/jira/browse/HDFS-11896?filter=12340814
>>>>> Would be good if people could follow up on review comments.
>>>>>
>>>>> I looked through nightly Jenkins build results for 2.7.4 both on
Apache
>>>>> Jenkins and internal.
>>>>> Some test fail intermittently, but there no consistent failures. I
>>>> filed
>>>>> HDFS-11985 to track some of them.
>>>>> https://issues.apache.org/jira/browse/HDFS-11985
>>>>> I do not currently consider these failures as blockers. LMK if some of
>>>>> them are.
>>>>>
>>>>> We started internal testing of branch-2.7 on one of our smallish (100+
>>>>> nodes) test clusters.
>>>>> Will update on the results.
>>>>>
>>>>> There is a plan to enable BigTop for 2.7.4 testing.
>>>>>
>>>>> Akira, Brahma thank you for setting up a wiki page for 2.7.4 release.
>>>>> Thank you everybody for contributing to this effort.
>>>>>
>>>>> Regards,
>>>>> --Konstantin
>>>>>
>>>>>
>>>>> On Tue, May 30, 2017 at 12:08 AM, Akira Ajisaka <aajis...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> Sure.
>>>>>> If you want to edit the wiki, please tell me your ASF confluence
>>>> account.
>>>>>>
>>>>>> -Akira
>>>>>>
>>>>>> On 2017/05/30 15:31, Rohith Sharma K S wrote:
>>>>>>
>>>>>>> Couple of more JIRAs need to be back ported for 2.7.4 release. These
>>>> will
>>>>>>> solve RM HA unstability issues.
>>>>>>> https://issues.apache.org/jira/browse/YARN-5333
>>>>>>> https://issues.apache.org/jira/browse/YARN-5988
>>>>>>> https://issues.apache.org/jira/browse/YARN-6304
>>>>>>>
>>>>>>> I will raise a JIRAs to back port it.
>>>>>>>
>>>>>>> @Akira , could you help to add these JIRAs into wiki?
>>>>>>>
>>>>>>> Thanks & Regards
>>>>>>> Rohith Sharma K S
>>>>>>>
>>>>>>> On 29 May 2017 at 12:19, Akira Ajisaka <aajis...@apache.org> wrote:
>>>>>>>
>>>>>>> Created a page for 2.7.4 release.
>>>>>>>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.7.4
>>>>>>>>
>>>>>>>> If you want to edit this wiki, please ping me.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Akira
>>>>>>>>
>>>>>>>>
>>>>>>>> On 2017/05/23 4:42, Brahma Reddy Battula wrote:
>>>>>>>>
>>>>>>>> Hi Konstantin Shvachko
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> how about creating a wiki page for 2.7.4 release status like 2.8
>>>> and
>>>>>>>>> trunk in following link.??
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://cwiki.apache.org/confluence/display/HADOOP
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ________________________________
>>>>>>>>> From: Konstantin Shvachko <shv.had...@gmail.com>
>>>>>>>>> Sent: Saturday, May 13, 2017 3:58 AM
>>>>>>>>> To: Akira Ajisaka
>>>>>>>>> Cc: Hadoop Common; Hdfs-dev; mapreduce-...@hadoop.apache.org;
>>>>>>>>> yarn-...@hadoop.apache.org
>>>>>>>>> Subject: Re: About 2.7.4 Release
>>>>>>>>>
>>>>>>>>> Latest update on the links and filters. Here is the correct link
>>>> for
>>>>>>>>> the
>>>>>>>>> filter:
>>>>>>>>> https://issues.apache.org/jira/secure/IssueNavigator.jspa?
>>>>>>>>> requestId=12340814
>>>>>>>>>
>>>>>>>>> Also updated: https://s.apache.org/Dzg4
>>>>>>>>>
>>>>>>>>> Had to do some Jira debugging. Sorry for confusion.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> --Konstantin
>>>>>>>>>
>>>>>>>>> On Wed, May 10, 2017 at 2:30 PM, Konstantin Shvachko <
>>>>>>>>> shv.had...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>> Hey Akira,
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I didn't have private filters. Most probably Jira caches
>>>> something.
>>>>>>>>>> Your filter is in the right direction, but for some reason it
>>>> lists
>>>>>>>>>> only
>>>>>>>>>> 22 issues, while mine has 29.
>>>>>>>>>> It misses e.g. YARN-5543 <https://issues.apache.org/jir
>>>>>>>>>> a/browse/YARN-5543>
>>>>>>>>>> .
>>>>>>>>>>
>>>>>>>>>> Anyways, I created a Jira filter now "Hadoop 2.7.4 release
>>>> blockers",
>>>>>>>>>> shared it with "everybody", and updated my link to point to that
>>>>>>>>>> filter.
>>>>>>>>>> So
>>>>>>>>>> you can use any of the three methods below to get the correct
>>>> list:
>>>>>>>>>> 1. Go to https://s.apache.org/Dzg4
>>>>>>>>>> 2. Go to the filter via
>>>>>>>>>> https://issues.apache.org/jira/issues?filter=12340814
>>>>>>>>>> or by finding "Hadoop 2.7.4 release blockers" filter in the
>>>> jira
>>>>>>>>>> 3. On Advanced issues search page paste this:
>>>>>>>>>> project in (HDFS, HADOOP, YARN, MAPREDUCE) AND labels =
>>>>>>>>>> release-blocker
>>>>>>>>>> AND "Target Version/s" = 2.7.4
>>>>>>>>>>
>>>>>>>>>> Hope this solves the confusion for which issues are included.
>>>>>>>>>> Please LMK if it doesn't, as it is important.
>>>>>>>>>>
>>>>>>>>>> Thanks,
>>>>>>>>>> --Konstantin
>>>>>>>>>>
>>>>>>>>>> On Tue, May 9, 2017 at 9:58 AM, Akira Ajisaka <
>>>> aajis...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Hi Konstantin,
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Thank you for volunteering as release manager!
>>>>>>>>>>>
>>>>>>>>>>> Actually the original link works fine: https://s.apache.org/Dzg4
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I couldn't see the link. Maybe is it private filter?
>>>>>>>>>>>
>>>>>>>>>>> Here is a link I generated: https://s.apache.org/ehKy
>>>>>>>>>>> This filter includes resolved issue and excludes fixversion ==
>>>> 2.7.4
>>>>>>>>>>>
>>>>>>>>>>> Thanks and Regards,
>>>>>>>>>>> Akira
>>>>>>>>>>>
>>>>>>>>>>> On 2017/05/08 19:20, Konstantin Shvachko wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi Brahma Reddy Battula,
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Actually the original link works fine:
>>>> https://s.apache.org/Dzg4
>>>>>>>>>>>> Your link excludes closed and resolved issues, which needs
>>>>>>>>>>>> backporting,
>>>>>>>>>>>> and
>>>>>>>>>>>> which we cannot reopen, as discussed in this thread earlier.
>>>>>>>>>>>>
>>>>>>>>>>>> Looked through the issues you proposed:
>>>>>>>>>>>>
>>>>>>>>>>>> HDFS-9311 <https://issues.apache.org/jira/browse/HDFS-9311>
>>>>>>>>>>>> Seems like a new feature. It helps failover to standby node
when
>>>>>>>>>>>> primary
>>>>>>>>>>>> is
>>>>>>>>>>>> under heavy load, but it introduces new APIs, addresses, config
>>>>>>>>>>>> parameters.
>>>>>>>>>>>> And needs at least one follow up jira.
>>>>>>>>>>>> Looks like a backward compatible change, though.
>>>>>>>>>>>> Did you have a chance to run it in production?
>>>>>>>>>>>>
>>>>>>>>>>>> +1 on
>>>>>>>>>>>> HDFS-10987 <https://issues.apache.org/jira/browse/HDFS-10987>
>>>>>>>>>>>>
>>>>>>>>>>>> [HDFS-10987] Make Decommission less expensive when lot of ...<
>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/HDFS-10987>
>>>>>>>>> issues.apache.org
>>>>>>>>> When user want to decommission a node which having 50M blocks ,it
>>>> could
>>>>>>>>> hold the namesystem lock for long time.We've seen it is taking 36
>>>> sec.
>>>>>>>>> As
>>>>>>>>> we knew during this ...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> HDFS-9902 <https://issues.apache.org/jira/browse/HDFS-9902>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> [HDFS-9902] Support different values of dfs.datanode.du ...<
>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/HDFS-9902>
>>>>>>>>> issues.apache.org
>>>>>>>>> Now Hadoop support different storage type for DISK, SSD, ARCHIVE
>>>> and
>>>>>>>>> RAM_DISK, but they share one configuration
>>>> dfs.datanode.du.reserved.
>>>>>>>>> The
>>>>>>>>> DISK size may be several ...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> HDFS-8312 <https://issues.apache.org/jira/browse/HDFS-8312>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> Trash does not descent into child directories to check for ...<
>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/HDFS-8312>
>>>>>>>>> issues.apache.org
>>>>>>>>> HDFS trash does not descent into child directory to check if user
>>>> has
>>>>>>>>> permission to delete files. For example: Run the following command
>>>> to
>>>>>>>>> initialize directory ...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> HADOOP-14100 <https://issues.apache.org/jira/browse/HADOOP-14100>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>> Upgrade Jsch jar to latest version to fix vulnerability in ...<
>>>>>>>>>>>
>>>>>>>>>> https://issues.apache.org/jira/browse/HADOOP-14100>
>>>>>>>>> issues.apache.org
>>>>>>>>> Recently there was on vulnerability reported on jsch library. Its
>>>> fixed
>>>>>>>>> in latest 0.1.54 version before CVE was made public.
>>>>>>>>> https://cve.mitre.org/cgi-bin/cvename.cgi ...
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Added them to 2.7.4 release. You should see them via the above
link
>>>>>>>>>>>> now.
>>>>>>>>>>>> Would be good if you could attach backport patches for some of
>>>> them?
>>>>>>>>>>>>
>>>>>>>>>>>> Appreciate your help,
>>>>>>>>>>>> --Konstantin
>>>>>>>>>>>>
>>>>>>>>>>>> On Mon, May 8, 2017 at 8:39 AM, Brahma Reddy Battula <
>>>>>>>>>>>> brahmareddy.batt...@huawei.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> Looks following link is not correct..
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://s.apache.org/Dzg4
>>>>>>>>>>>>>
>>>>>>>>>>>>> It should be like following..?
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://s.apache.org/wi3U
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Apart from Konstantin mentioned,Following also good to go..?
>>>> let me
>>>>>>>>>>>>> know
>>>>>>>>>>>>> your thoughts on this.
>>>>>>>>>>>>>
>>>>>>>>>>>>> For Large Cluster:
>>>>>>>>>>>>> =============
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HDFS-9311===Life Line
>>>>>>>>>>>>> Protocol
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HDFS-10987===Deecommis
>>>> sion
>>>>>>>>>>>>> Expensive when lot's of blocks are present
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HDFS-9902===
>>>>>>>>>>>>> "dfs.datanode.du.reserved" per Storage Type
>>>>>>>>>>>>>
>>>>>>>>>>>>> For Security:
>>>>>>>>>>>>> =========
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HDFS-8312===Trash does
>>>> not
>>>>>>>>>>>>> descent
>>>>>>>>>>>>> into child directories to check for permission
>>>>>>>>>>>>> https://issues.apache.org/jira/browse/HADOOP-14100===Upgrade
>>>> Jsch
>>>>>>>>>>>>> jar
>>>>>>>>>>>>> to
>>>>>>>>>>>>> latest version to fix vulnerability in old versions
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Brahma Reddy Battula
>>>>>>>>>>>>>
>>>>>>>>>>>>> -----Original Message-----
>>>>>>>>>>>>> From: Erik Krogen [mailto:ekro...@linkedin.com.INVALID]
>>>>>>>>>>>>> Sent: 06 May 2017 02:40
>>>>>>>>>>>>> To: Konstantin Shvachko
>>>>>>>>>>>>> Cc: Zhe Zhang; Hadoop Common; Hdfs-dev;
>>>>>>>>>>>>> mapreduce-...@hadoop.apache.org
>>>>>>>>>>>>> ;
>>>>>>>>>>>>> yarn-...@hadoop.apache.org
>>>>>>>>>>>>> Subject: Re: About 2.7.4 Release
>>>>>>>>>>>>>
>>>>>>>>>>>>> List LGTM Konstantin!
>>>>>>>>>>>>>
>>>>>>>>>>>>> Let's say that we will only create a new tracking JIRA for
>>>> patches
>>>>>>>>>>>>> which
>>>>>>>>>>>>> do not backport cleanly, to avoid having too many lying
around.
>>>>>>>>>>>>> Otherwise
>>>>>>>>>>>>> we can directly attach to old ticket. If a clean backport does
>>>>>>>>>>>>> happen
>>>>>>>>>>>>> to
>>>>>>>>>>>>> break a test the nightly build will help us catch it.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Erik
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Thu, May 4, 2017 at 7:21 PM, Konstantin Shvachko <
>>>>>>>>>>>>> shv.had...@gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>> Great Zhe. Let's monitor the build.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> I marked all jiras I knew of for inclusion into 2.7.4 as I
>>>>>>>>>>>>>> described
>>>>>>>>>>>>>> before.
>>>>>>>>>>>>>> Target Version/s: 2.7.4
>>>>>>>>>>>>>> Label: release-blocker
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Here is the link to the list: https://s.apache.org/Dzg4
>>>> Please
>>>>>>>>>>>>>> let
>>>>>>>>>>>>>> me
>>>>>>>>>>>>>> know if I missed anything.
>>>>>>>>>>>>>> And feel free to pick up any. Most of backports are pretty
>>>>>>>>>>>>>> straightforward, but not all.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We can create tracking jiras for backporting if you need to
>>>> run
>>>>>>>>>>>>>> Jenkins on the patch (and since Allen does not allow
reopening
>>>>>>>>>>>>>> them).
>>>>>>>>>>>>>> But I think the final patch should be attached to the
original
>>>>>>>>>>>>>> jira.
>>>>>>>>>>>>>> Otherwise history will be hard to follow.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>> --Konstantin
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, May 3, 2017 at 4:53 PM, Zhe Zhang <z...@apache.org>
>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks for volunteering as RM Konstantin! The plan LGTM.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I've created a nightly Jenkins job for branch-2.7 (unit
>>>> tests):
>>>>>>>>>>>>>>> https://builds.apache.org/job/Hadoop-branch2.7-nightly/
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, May 3, 2017 at 12:42 AM Konstantin Shvachko <
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> shv.had...@gmail.com>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hey guys,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I and a few of my colleagues would like to help here and
>>>> move
>>>>>>>>>>>>>>>> 2.7.4
>>>>>>>>>>>>>>>> release forward. A few points in this regard.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 1. Reading through this thread since March 1 I see that
>>>> Vinod
>>>>>>>>>>>>>>>> hinted on managing the release. Vinod, if you still want
the
>>>>>>>>>>>>>>>> job /
>>>>>>>>>>>>>>>> have bandwidth will be happy to work with you.
>>>>>>>>>>>>>>>> Otherwise I am glad to volunteer as the release manager.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2. In addition to current blockers and criticals, I would
>>>> like
>>>>>>>>>>>>>>>> to
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> propose
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> a
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> few issues to be included in the release, see the list
below.
>>>>>>>>>>>>>>>> Those
>>>>>>>>>>>>>>>> are mostly bug fixes and optimizations, which we already
>>>> have in
>>>>>>>>>>>>>>>> our
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> internal
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> branch and run in production. Plus one minor feature "node
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> labeling", which we found very handy, when you have
>>>> heterogeneous
>>>>>>>>>>>>>>>> environments and mixed workloads, like MR and Spark.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 3. For marking issues for the release I propose to
>>>>>>>>>>>>>>>> - set the target version to 2.7.4, and
>>>>>>>>>>>>>>>> - add a new label "release-blocker"
>>>>>>>>>>>>>>>> That way we will know issues targeted for the release
>>>> without
>>>>>>>>>>>>>>>> reopening them for backports.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 4. I see quite a few people are interested in the release.
>>>> With
>>>>>>>>>>>>>>>> all
>>>>>>>>>>>>>>>> the help I think we can target to release by the end of
May.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Other things include fixing CHANGES.txt and fixing Jenkins
>>>> build
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2.7.4
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> branch.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>>>>> --Konstantin
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ========== List of issue for 2.7.4 ===========
>>>>>>>>>>>>>>>> ------ Backports
>>>>>>>>>>>>>>>> HADOOP-12975 <https://issues.apache.org/jir
>>>>>>>>>>>>>>>> a/browse/HADOOP-12975>.
>>>>>>>>>>>>>>>> Add
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> du
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> jitters
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> HDFS-9710 <https://issues.apache.org/jira/browse/HDFS-9710>.
>>>> IBR
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> batching
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> HDFS-10715 <https://issues.apache.org/jira/browse/HDFS-10715
>>>>> .
>>>>>>>>>>>>>> NPE
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> when applying AvailableSpaceBlockPlacementPolicy
>>>>>>>>>>>>>>>> HDFS-2538 <https://issues.apache.org/jira/browse/HDFS-2538
>>>>> .
>>>>>>>>>>>>>>>> fsck
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> removal
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> of dot printing
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> HDFS-8131 <https://issues.apache.org/jira/browse/HDFS-8131>.
>>>>>>>>>>>>>>>> space-balanced
>>>>>>>>>>>>>>>> policy for balancer
>>>>>>>>>>>>>>>> HDFS-8549 <https://issues.apache.org/jira/browse/HDFS-8549
>>>>> .
>>>>>>>>>>>>>>>> abort
>>>>>>>>>>>>>>>> balancer if upgrade in progress
>>>>>>>>>>>>>>>> HDFS-9412 <https://issues.apache.org/jira/browse/HDFS-9412
>>>>> .
>>>>>>>>>>>>>>>> skip
>>>>>>>>>>>>>>>> small blocks in getBlocks
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> YARN-1471 <https://issues.apache.org/jira/browse/YARN-1471
>>>>> .
>>>>>>>>>>>>>>>> SLS
>>>>>>>>>>>>>>>> simulator
>>>>>>>>>>>>>>>> YARN-4302 <https://issues.apache.org/jira/browse/YARN-4302
>>>>> .
>>>>>>>>>>>>>>>> SLS
>>>>>>>>>>>>>>>> YARN-4367 <https://issues.apache.org/jira/browse/YARN-4367
>>>>> .
>>>>>>>>>>>>>>>> SLS
>>>>>>>>>>>>>>>> YARN-4612 <https://issues.apache.org/jira/browse/YARN-4612
>>>>> .
>>>>>>>>>>>>>>>> SLS
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ----- Node labeling
>>>>>>>>>>>>>>>> MAPREDUCE-6304
>>>>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/MAPREDUCE-6304>
>>>>>>>>>>>>>>>> YARN-2943 <https://issues.apache.org/jira/browse/YARN-2943>
>>>>>>>>>>>>>>>> YARN-4109 <https://issues.apache.org/jira/browse/YARN-4109>
>>>>>>>>>>>>>>>> YARN-4140 <https://issues.apache.org/jira/browse/YARN-4140>
>>>>>>>>>>>>>>>> YARN-4250 <https://issues.apache.org/jira/browse/YARN-4250>
>>>>>>>>>>>>>>>> YARN-4925 <https://issues.apache.org/jira/browse/YARN-4925>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> --
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Zhe Zhang
>>>>>>>>>>>>>>> Apache Hadoop Committer
>>>>>>>>>>>>>>> http://zhe-thoughts.github.io/about/ | @oldcap
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------
>>>> ---------
>>>>>>>> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>>>>>>>> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>
>>>>
>>>
>>
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
