Re: When are incompatible changes acceptable (HDFS-12990)

[Tsz Wo (Nicholas), Sze]

 (Re-sent. Just found that my previous email seems not delivered to common-dev.)
   
> > The question is: how are we going to fix it?>> What do you propose? -C
First of all, let's state clearly what is the problem about.  Please help me 
out if I have missed anything.
The problem reported by HDFS-12990 is that HDFS-9427 has changed NN default RPC 
port from 8020 to 9820.  HDFS-12990 claimed, “the NN RPC port change is painful 
for downstream on migrating to Hadoop 3.”
Note 1: This isn't a problem for HA cluster.Note 2: The port is configurable.  
User can set it to any value.Note 3: HDFS-9427 has also changed many other 
HTTP/RPC ports as shown below
Namenode ports: 50470 --> 9871, 50070 --> 9870, 8020 --> 9820Secondary NN 
ports: 50091 --> 9869, 50090 --> 9868Datanode ports: 50020 --> 9867, 50010 --> 
9866, 50475 --> 9865, 50075 --> 9864 
The other port changes probably also affect downstream projects and give them a 
“painful” experience.  For example, NN UI and WebHDFS use a different port.
The problem is related convenience but not anything serious like a security bug.
There are a few possible solutions:1) Considered that the port changes are not 
limited to NN RPC and the default port value should not be hardcoded.  Also, 
downstream projects probably need to fix other hardcoded ports (e.g. WebHDFS) 
anyway.  Let’s just keep all the port changes and document them clearly about 
the changes (we may throw an exception if some applications try to connect to 
the old ports.)  In this way, 3.0.1 is compatible with 3.0.0.
2) Further change the NN RPC so that NN listens to both 8020 and 9820 by 
default.  It is a new feature that NN listen to two ports simultaneously.  The 
feature has other benefits, e.g. one of the ports is reserved to some high 
priority applications so that it can have a better response time.  It is 
compatible to both 2.x and 3.0.0. Of course, users could choose to set it back 
to one of the ports in the conf.
3) Revert the NN RPC port back to 8020.  We need to ask where should the revert 
happen?3.1) Revert it in 3.0.1 as proposed by HDFS-12990.  However, this is an 
incompatible change between dot releases 3.0.0 and 3.0.1 and it violates our 
policy.  Being compatible is very important.  Users expect 3.0.0 and 3.0.1 are 
compatible.  How could we explain 3.0.0 and 3.0.1 are incompatible due to 
convenience?3.2) Revert it in 4.0.0.  There is no compatibility issue since 
3.0.0 and 4.0.0 are allowed to have incompatible changes according to our 
policy.
Since compatibility is more important than convenience, Solution #3.1 is 
impermissible.  For the remaining solutions, both #1 and #2 are fine to me.
Thanks.Tsz-Wo


On Friday, January 12, 2018, 12:26:47 PM GMT+8, Chris Douglas 
 wrote:  
On Thu, Jan 11, 2018 at 6:34 PM Tsz Wo Sze  wrote:

 
The question is: how are we going to fix it?


What do you propose? -C




> No incompatible changes are allowed between 3.0.0 and 3.0.1. Dot releases 
> only allow bug fixes.

We may not like the statement above but it is our compatibility policy.  We 
should either follow the policy or revise it.

Some more questions:
   
   - What if someone is already using 3.0.0 and has changed all the scripts to 
9820?  Just let them fail?
   - Compared to 2.x, 3.0.0 has many incompatible changes. Are we going to have 
other incompatible changes in the future minor and dot releases? What is the 
criteria to decide which incompatible changes are allowed?
   - I hate that we have prematurely released 3.0.0 and make 3.0.1 incompatible 
to 3.0.0. If the "bug" is that serious, why not fixing it in 4.0.0 and declare 
3.x as dead?
   - It seems obvious that no one has seriously tested it so that the problem 
is not uncovered until now. Are there bugs in our current release procedure?

ThanksTsz-Wo


On Thursday, January 11, 2018, 11:36:33 AM GMT+8, Chris Douglas 
 wrote:  
 
 Isn't this limited to reverting the 8020 -> 9820 change? -C

On Wed, Jan 10, 2018 at 6:13 PM Eric Yang  wrote:

> The fix in HDFS-9427 can potentially bring in new customers because less
> chance for new comer to encountering “port already in use” problem.  If we
> make change according to HDFS-12990, then this incompatible change does not
> make incompatible change compatible.  Other ports are not reverted
> according to HDFS-12990.  User will encounter the bad taste in the mouth
> that HDFS-9427 attempt to solve.  Please do consider both negative side
> effects of reverting as well as incompatible minor release change.  Thanks
>
> Regards,
> Eric
>
> From: larry mccay 
> Date: Wednesday, January 10, 2018 at 10:53 AM
> To: Daryn Sharp 
> Cc: "Aaron T. Myers" , Eric Yang ,
> Chris Douglas , Hadoop Common <
> common-dev@hadoop.apache.org>
> Subject: Re: When are incompatible changes acceptable (HDFS-12990)
>
> 

Re: Hadoop 3.1.0 release discussion

[Daniel Templeton]

What's the status on resource profiles?  I believe there are still a 
couple of open JIRAs to rethink some of the design choices.


Daniel

On 1/17/18 11:33 AM, Wangda Tan wrote:

Hi All,

Since we're fast approaching previously proposed feature freeze date (Jan
30, about 13 days from today). If you've any features which live in a
branch and targeted to 3.1.0, please reply this email thread. Ideally, we
should finish branch merging before feature freeze date.

Here's an updated 3.1.0 feature status:

1. Merged & Completed features:
* (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
* (Wangda) YARN-6223: GPU support on YARN. Features in trunk and works
end-to-end.
* (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
* (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename commits.
* (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation of Leaf
Queues While Doing Queue Mapping.
* (Chris Douglas) HDFS-9806: HDFS Tiered Storage.

2. Features close to finish:
* (Zhankun) YARN-5983: FPGA support. Majority implementations completed and
merged to trunk. Except for UI/documentation.
* (Uma) HDFS-10285: HDFS SPS. Majority implementations are done, some
discussions going on about implementation.
* (Arun Suresh / Kostas / Wangda). YARN-6592: New SchedulingRequest and
anti-affinity support. Close to finish, on track to be merged before Jan 30.

3. Tentative features:
* (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
containers. Only one pending patch. Plan to finish before Jan 7th.
* (Haibo Chen). YARN-1011: Resource overcommitment. Looks challenging to be
done before Jan 2018.
* (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240. Looks
challenging to be done before Jan 2018.
* (Varun V) YARN-5673: container-executor write. Given security refactoring
of c-e (YARN-6623) is already landed, IMHO other stuff may be moved to 3.2.

Thanks,
Wangda




On Fri, Dec 15, 2017 at 1:20 PM, Wangda Tan  wrote:


Hi all,

Congratulations on the 3.0.0-GA release!

As we discussed in the previous email thread [1], I'd like to restart
3.1.0 release plans.

a) Quick summary:
a.1 Release status
We started 3.1 release discussion on Sep 6, 2017 [1]. As of today,
there’re 232 patches loaded on 3.1.0 alone [2], besides 6 open blockers and
22 open critical issues.

a.2 Release date update
Considering delays of 3.0-GA release by month-and-a-half, I propose to
move the dates as follows
  - feature freeze date from Dec 15, 2017, to Jan 30, 2018 - last date for
any branches to get merged too;
  - code freeze (blockers & critical only) date to Feb 08, 2018;
  - release voting start by Feb 18, 2018, leaving time for at least two RCx
  - release date from Jan 15, 2018, to Feb 28, 2018;

Unlike before, I added an additional milestone for release-vote-start so
that we can account for voting time-period also.

This overall is still 5 1/2 months of release-timeline unlike the faster
cadence we hoped for, but this, in my opinion, is the best-updated timeline
given the delays of the final release of 3.0-GA.

b) Individual feature status:
I spoke to several feature owners and checked the status of un-finished
features, following are status of features planned to 3.1.0:

b.1 Merged & Completed features:
* (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
* (Wangda) YARN-6223: GPU support on YARN. Features in trunk and works
end-to-end.
* (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
* (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename
commits.
* (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation of Leaf
Queues While Doing Queue Mapping.

b.2 Features close to finish:
* (Chris Douglas) HDFS-9806: HDFS Tiered Storage. Being voting now.
* (Zhankun) YARN-5983: FPGA support. Majority implementations completed
and merged to trunk. Except for UI/documentation.
* (Uma) HDFS-10285: HDFS SPS. Majority implementations are done, some
discussions going on about implementation.

b.3 Tentative features:
* (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
containers. Only one pending patch. Plan to finish before Jan 7th.
* (Haibo Chen). YARN-1011: Resource overcommitment. Looks challenging to
be done before Jan 2018.
* (Arun Suresh / Kostas / Wangda). YARN-6592: New SchedulingRequest and
anti-affinity support. Tentative will figure out by Jan 1st.
* (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240. Looks
challenging to be done before Jan 2018.
* (Varun V) YARN-5673: container-executor write. Given security
refactoring of c-e (YARN-6623) is already landed, IMHO other stuff may be
moved to 3.2.

b.4 Additional release drivers
* More exhaustive upgrade testing from 2.x to 3.x.

c) Regarding branch cut:

We will keep pointing trunk to 3.1 and cut branch-3.1 until: A. some
feature planned to 3.2 has to be landed on trunk or B. After feature freeze
date, whichever comes first.

I've also talked 

Re: Hadoop 3.1.0 release discussion

[Gangumalla, Uma]

HI Wangda,

 Thank you for the head-up mail.
 We are in the branch (HDFS-10285) and trying to push the tasks sooner before 
the deadline.

Regards,
Uma

On 1/17/18, 11:35 AM, "Wangda Tan"  wrote:

Hi All,

Since we're fast approaching previously proposed feature freeze date (Jan
30, about 13 days from today). If you've any features which live in a
branch and targeted to 3.1.0, please reply this email thread. Ideally, we
should finish branch merging before feature freeze date.

Here's an updated 3.1.0 feature status:

1. Merged & Completed features:
* (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
* (Wangda) YARN-6223: GPU support on YARN. Features in trunk and works
end-to-end.
* (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
* (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename commits.
* (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation of Leaf
Queues While Doing Queue Mapping.
* (Chris Douglas) HDFS-9806: HDFS Tiered Storage.

2. Features close to finish:
* (Zhankun) YARN-5983: FPGA support. Majority implementations completed and
merged to trunk. Except for UI/documentation.
* (Uma) HDFS-10285: HDFS SPS. Majority implementations are done, some
discussions going on about implementation.
* (Arun Suresh / Kostas / Wangda). YARN-6592: New SchedulingRequest and
anti-affinity support. Close to finish, on track to be merged before Jan 30.

3. Tentative features:
* (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
containers. Only one pending patch. Plan to finish before Jan 7th.
* (Haibo Chen). YARN-1011: Resource overcommitment. Looks challenging to be
done before Jan 2018.
* (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240. Looks
challenging to be done before Jan 2018.
* (Varun V) YARN-5673: container-executor write. Given security refactoring
of c-e (YARN-6623) is already landed, IMHO other stuff may be moved to 3.2.

Thanks,
Wangda




On Fri, Dec 15, 2017 at 1:20 PM, Wangda Tan  wrote:

> Hi all,
>
> Congratulations on the 3.0.0-GA release!
>
> As we discussed in the previous email thread [1], I'd like to restart
> 3.1.0 release plans.
>
> a) Quick summary:
> a.1 Release status
> We started 3.1 release discussion on Sep 6, 2017 [1]. As of today,
> there’re 232 patches loaded on 3.1.0 alone [2], besides 6 open blockers 
and
> 22 open critical issues.
>
> a.2 Release date update
> Considering delays of 3.0-GA release by month-and-a-half, I propose to
> move the dates as follows
>  - feature freeze date from Dec 15, 2017, to Jan 30, 2018 - last date for
> any branches to get merged too;
>  - code freeze (blockers & critical only) date to Feb 08, 2018;
>  - release voting start by Feb 18, 2018, leaving time for at least two RCx
>  - release date from Jan 15, 2018, to Feb 28, 2018;
>
> Unlike before, I added an additional milestone for release-vote-start so
> that we can account for voting time-period also.
>
> This overall is still 5 1/2 months of release-timeline unlike the faster
> cadence we hoped for, but this, in my opinion, is the best-updated 
timeline
> given the delays of the final release of 3.0-GA.
>
> b) Individual feature status:
> I spoke to several feature owners and checked the status of un-finished
> features, following are status of features planned to 3.1.0:
>
> b.1 Merged & Completed features:
> * (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
> * (Wangda) YARN-6223: GPU support on YARN. Features in trunk and works
> end-to-end.
> * (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
> * (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename
> commits.
> * (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation of Leaf
> Queues While Doing Queue Mapping.
>
> b.2 Features close to finish:
> * (Chris Douglas) HDFS-9806: HDFS Tiered Storage. Being voting now.
> * (Zhankun) YARN-5983: FPGA support. Majority implementations completed
> and merged to trunk. Except for UI/documentation.
> * (Uma) HDFS-10285: HDFS SPS. Majority implementations are done, some
> discussions going on about implementation.
>
> b.3 Tentative features:
> * (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
> containers. Only one pending patch. Plan to finish before Jan 7th.
> * (Haibo Chen). YARN-1011: Resource overcommitment. Looks challenging to
> be done before Jan 2018.
> * (Arun Suresh / Kostas / Wangda). YARN-6592: New SchedulingRequest and
> anti-affinity support. Tentative will figure out by Jan 1st.
> * (Anu): 

[jira] [Reopened] (HADOOP-12751) While using kerberos Hadoop incorrectly assumes names with '@' to be non-simple

[Konstantin Shvachko (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-12751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Konstantin Shvachko reopened HADOOP-12751:
--

> While using kerberos Hadoop incorrectly assumes names with '@' to be 
> non-simple
> ---
>
> Key: HADOOP-12751
> URL: https://issues.apache.org/jira/browse/HADOOP-12751
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: security
>Affects Versions: 2.7.2
> Environment: kerberos
>Reporter: Bolke de Bruin
>Assignee: Bolke de Bruin
>Priority: Critical
>  Labels: kerberos
> Fix For: 2.8.0, 3.0.0-alpha1
>
> Attachments: 0001-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0001-Remove-check-for-user-name-characters-and.patch, 
> 0002-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0003-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0004-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0005-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0006-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0007-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0007-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0008-HADOOP-12751-leave-user-validation-to-os.patch, 
> 0008-HADOOP-12751-leave-user-validation-to-os.patch, HADOOP-12751-009.patch
>
>
> In the scenario of a trust between two directories, eg. FreeIPA (ipa.local) 
> and Active Directory (ad.local) users can be made available on the OS level 
> by something like sssd. The trusted users will be of the form 'user@ad.local' 
> while other users are will not contain the domain. Executing 'id -Gn 
> user@ad.local' will successfully return the groups the user belongs to if 
> configured correctly. 
> However, it is assumed by Hadoop that users of the format with '@' cannot be 
> correct. This code is in KerberosName.java and seems to be a validator if the 
> 'auth_to_local' rules are applied correctly.
> In my opinion this should be removed or changed to a different kind of check 
> or maybe logged as a warning while still proceeding, as the current behavior 
> limits integration possibilities with other standard tools.
> Workaround are difficult to apply (by having a rewrite by system tools to for 
> example user_ad_local) due to down stream consequences.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15178) Generalize NetUtils#wrapException to handle other subclasses with String Constructor.

[Ajay Kumar (JIRA)]

Ajay Kumar created HADOOP-15178:
---

 Summary: Generalize NetUtils#wrapException to handle other 
subclasses with String Constructor.
 Key: HADOOP-15178
 URL: https://issues.apache.org/jira/browse/HADOOP-15178
 Project: Hadoop Common
  Issue Type: Bug
 Environment: NetUtils#wrapException returns an IOException if 
exception passed to it is not of type 
SocketException,EOFException,NoRouteToHostException,SocketTimeoutException,UnknownHostException,ConnectException,BindException.
By default, it  should always return instance (subclass of IOException) of same 
type unless a String constructor is not available.
Reporter: Ajay Kumar
Assignee: Ajay Kumar






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: What's the difference between branch-3 and branch-3.0?

[Jason Lowe]

I filed INFRA-15859 to have branch-3 deleted.

Jason

On Wed, Jan 17, 2018 at 1:23 PM, Eric Payne <
eric.payne1...@yahoo.com.invalid> wrote:

> +1 for removing the branch-3 branch. It should be done soon so more
> confusion can be avoided. Thanks Jason for tracking this down.
> -Eric
>
>
>   From: Jason Lowe 
>  To: Hadoop Common ; ma...@cloudera.com
>  Sent: Wednesday, January 17, 2018 12:42 PM
>  Subject: Re: What's the difference between branch-3 and branch-3.0?
>
> This was created accidentally when HDFS-11847 was committed.  As such we
> should delete the branch-3 branch and port over the commits that went into
> branch-3 instead of branch-3.0.  For the former, I'm assuming that requires
> an INFRA ticket since I would hope any committer would not have the ability
> to destroy a branch-* branch.  Unfortunately even after it's deleted I
> suspect we will see it reappear if someone pushes up their old copy of
> branch-3 again, so committers will need to be vigilant.
>
> I'll work on porting the missing changes below from branch-3 over to
> branch-3.0.  I'll wait for some more consensus on the branch-3 deletion
> before filing the INFRA ticket since deleting a branch shouldn't be done
> lightly.
>
> Jason
>
>
> commit 0802d8afa355d9a0683fdb2e9c4963e8fea8644f
> Author: Vinayakumar B 
> Date:  Wed Jan 17 14:16:48 2018 +0530
>
> HDFS-9049. Make Datanode Netty reverse proxy port to be configurable.
> Contributed by Vinayakumar B.
>
> (cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
>
> commit db8345fa9cd124728d935f725525e2626438b4c1
> Author: Lei Xu 
> Date:  Tue Jan 16 15:15:11 2018 -0800
>
> HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when
> safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei)
>
> (cherry picked from commit 3bd9ea63df769345a9d02a404cfb61323a4cd7e3)
>
> commit 82741091a78d7ce62c240ec3e7f81a3a9a3fee36
> Author: Inigo Goiri 
> Date:  Mon Jan 15 12:21:24 2018 -0800
>
> HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer.
> Contributed by Inigo Goiri.
>
> commit d3fbcd92fe53192a319683b9ac72179cb28bd978
> Author: Yiqun Lin 
> Date:  Sat Jan 6 14:31:08 2018 +0800
>
> HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under
> a given path. Contributed by Yiqun Lin.
>
> commit ee44783515a55ab9fd368660c5cc2c2bc392132e
> Author: Manoj Govindassamy 
> Date:  Tue Jan 2 14:59:36 2018 -0800
>
> HDFS-11847. Enhance dfsadmin listOpenFiles command to list files
> blocking datanode decommissioning.
>
>
>
> On Wed, Jan 17, 2018 at 10:53 AM, Brahma Reddy Battula 
> wrote:
>
> > IMHO,we no need to have *branch-3* still trunk moves.Shall we remove as
> it
> > make confuses.??
> >
> >
> >
> >
> > Brahma Reddy Battula
> >
> > On Wed, Jan 17, 2018 at 9:41 PM, Jason Lowe 
> > wrote:
> >
> > > I recently noticed some committers posting commits to branch-3 and
> > marking
> > > the JIRA as fixed in 3.0.1.  I thought branch-3.0 was tracking 3.0.x
> > > releases, including branch-3.0.1 as of now, so I am confused what
> > branch-3
> > > is for.  The versions in the poms between branch-3 and branch-3.0 both
> > say
> > > they are 3.0.1-SNAPSHOT.
> > >
> > > I recall we discussed _not_ creating branch-3 until it is necessary,
> and
> > it
> > > is only necessary for branch-3 to exist when trunk stop tracking 3.x
> > > releases (i.e.: when trunk moves to 4.0.0-SNAPSHOT).
> > >
> > > Jason
> > >
> >
> >
> >
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> >
>
>
>
>

Re: Hadoop 3.1.0 release discussion

[Wangda Tan]

Hi All,

Since we're fast approaching previously proposed feature freeze date (Jan
30, about 13 days from today). If you've any features which live in a
branch and targeted to 3.1.0, please reply this email thread. Ideally, we
should finish branch merging before feature freeze date.

Here's an updated 3.1.0 feature status:

1. Merged & Completed features:
* (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
* (Wangda) YARN-6223: GPU support on YARN. Features in trunk and works
end-to-end.
* (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
* (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename commits.
* (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation of Leaf
Queues While Doing Queue Mapping.
* (Chris Douglas) HDFS-9806: HDFS Tiered Storage.

2. Features close to finish:
* (Zhankun) YARN-5983: FPGA support. Majority implementations completed and
merged to trunk. Except for UI/documentation.
* (Uma) HDFS-10285: HDFS SPS. Majority implementations are done, some
discussions going on about implementation.
* (Arun Suresh / Kostas / Wangda). YARN-6592: New SchedulingRequest and
anti-affinity support. Close to finish, on track to be merged before Jan 30.

3. Tentative features:
* (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
containers. Only one pending patch. Plan to finish before Jan 7th.
* (Haibo Chen). YARN-1011: Resource overcommitment. Looks challenging to be
done before Jan 2018.
* (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240. Looks
challenging to be done before Jan 2018.
* (Varun V) YARN-5673: container-executor write. Given security refactoring
of c-e (YARN-6623) is already landed, IMHO other stuff may be moved to 3.2.

Thanks,
Wangda




On Fri, Dec 15, 2017 at 1:20 PM, Wangda Tan  wrote:

> Hi all,
>
> Congratulations on the 3.0.0-GA release!
>
> As we discussed in the previous email thread [1], I'd like to restart
> 3.1.0 release plans.
>
> a) Quick summary:
> a.1 Release status
> We started 3.1 release discussion on Sep 6, 2017 [1]. As of today,
> there’re 232 patches loaded on 3.1.0 alone [2], besides 6 open blockers and
> 22 open critical issues.
>
> a.2 Release date update
> Considering delays of 3.0-GA release by month-and-a-half, I propose to
> move the dates as follows
>  - feature freeze date from Dec 15, 2017, to Jan 30, 2018 - last date for
> any branches to get merged too;
>  - code freeze (blockers & critical only) date to Feb 08, 2018;
>  - release voting start by Feb 18, 2018, leaving time for at least two RCx
>  - release date from Jan 15, 2018, to Feb 28, 2018;
>
> Unlike before, I added an additional milestone for release-vote-start so
> that we can account for voting time-period also.
>
> This overall is still 5 1/2 months of release-timeline unlike the faster
> cadence we hoped for, but this, in my opinion, is the best-updated timeline
> given the delays of the final release of 3.0-GA.
>
> b) Individual feature status:
> I spoke to several feature owners and checked the status of un-finished
> features, following are status of features planned to 3.1.0:
>
> b.1 Merged & Completed features:
> * (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
> * (Wangda) YARN-6223: GPU support on YARN. Features in trunk and works
> end-to-end.
> * (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native services.
> * (Steve Loughran): HADOOP-13786: S3Guard committer for zero-rename
> commits.
> * (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation of Leaf
> Queues While Doing Queue Mapping.
>
> b.2 Features close to finish:
> * (Chris Douglas) HDFS-9806: HDFS Tiered Storage. Being voting now.
> * (Zhankun) YARN-5983: FPGA support. Majority implementations completed
> and merged to trunk. Except for UI/documentation.
> * (Uma) HDFS-10285: HDFS SPS. Majority implementations are done, some
> discussions going on about implementation.
>
> b.3 Tentative features:
> * (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
> containers. Only one pending patch. Plan to finish before Jan 7th.
> * (Haibo Chen). YARN-1011: Resource overcommitment. Looks challenging to
> be done before Jan 2018.
> * (Arun Suresh / Kostas / Wangda). YARN-6592: New SchedulingRequest and
> anti-affinity support. Tentative will figure out by Jan 1st.
> * (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240. Looks
> challenging to be done before Jan 2018.
> * (Varun V) YARN-5673: container-executor write. Given security
> refactoring of c-e (YARN-6623) is already landed, IMHO other stuff may be
> moved to 3.2.
>
> b.4 Additional release drivers
> * More exhaustive upgrade testing from 2.x to 3.x.
>
> c) Regarding branch cut:
>
> We will keep pointing trunk to 3.1 and cut branch-3.1 until: A. some
> feature planned to 3.2 has to be landed on trunk or B. After feature freeze
> date, whichever comes first.
>
> I've also talked offline with Vinod to get help on release-management
> 

Re: What's the difference between branch-3 and branch-3.0?

[Eric Payne]

+1 for removing the branch-3 branch. It should be done soon so more confusion 
can be avoided. Thanks Jason for tracking this down.
-Eric


  From: Jason Lowe 
 To: Hadoop Common ; ma...@cloudera.com 
 Sent: Wednesday, January 17, 2018 12:42 PM
 Subject: Re: What's the difference between branch-3 and branch-3.0?
   
This was created accidentally when HDFS-11847 was committed.  As such we
should delete the branch-3 branch and port over the commits that went into
branch-3 instead of branch-3.0.  For the former, I'm assuming that requires
an INFRA ticket since I would hope any committer would not have the ability
to destroy a branch-* branch.  Unfortunately even after it's deleted I
suspect we will see it reappear if someone pushes up their old copy of
branch-3 again, so committers will need to be vigilant.

I'll work on porting the missing changes below from branch-3 over to
branch-3.0.  I'll wait for some more consensus on the branch-3 deletion
before filing the INFRA ticket since deleting a branch shouldn't be done
lightly.

Jason


commit 0802d8afa355d9a0683fdb2e9c4963e8fea8644f
Author: Vinayakumar B 
Date:  Wed Jan 17 14:16:48 2018 +0530

    HDFS-9049. Make Datanode Netty reverse proxy port to be configurable.
Contributed by Vinayakumar B.

    (cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)

commit db8345fa9cd124728d935f725525e2626438b4c1
Author: Lei Xu 
Date:  Tue Jan 16 15:15:11 2018 -0800

    HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when
safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei)

    (cherry picked from commit 3bd9ea63df769345a9d02a404cfb61323a4cd7e3)

commit 82741091a78d7ce62c240ec3e7f81a3a9a3fee36
Author: Inigo Goiri 
Date:  Mon Jan 15 12:21:24 2018 -0800

    HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer.
Contributed by Inigo Goiri.

commit d3fbcd92fe53192a319683b9ac72179cb28bd978
Author: Yiqun Lin 
Date:  Sat Jan 6 14:31:08 2018 +0800

    HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under
a given path. Contributed by Yiqun Lin.

commit ee44783515a55ab9fd368660c5cc2c2bc392132e
Author: Manoj Govindassamy 
Date:  Tue Jan 2 14:59:36 2018 -0800

    HDFS-11847. Enhance dfsadmin listOpenFiles command to list files
blocking datanode decommissioning.



On Wed, Jan 17, 2018 at 10:53 AM, Brahma Reddy Battula 
wrote:

> IMHO,we no need to have *branch-3* still trunk moves.Shall we remove as it
> make confuses.??
>
>
>
>
> Brahma Reddy Battula
>
> On Wed, Jan 17, 2018 at 9:41 PM, Jason Lowe 
> wrote:
>
> > I recently noticed some committers posting commits to branch-3 and
> marking
> > the JIRA as fixed in 3.0.1.  I thought branch-3.0 was tracking 3.0.x
> > releases, including branch-3.0.1 as of now, so I am confused what
> branch-3
> > is for.  The versions in the poms between branch-3 and branch-3.0 both
> say
> > they are 3.0.1-SNAPSHOT.
> >
> > I recall we discussed _not_ creating branch-3 until it is necessary, and
> it
> > is only necessary for branch-3 to exist when trunk stop tracking 3.x
> > releases (i.e.: when trunk moves to 4.0.0-SNAPSHOT).
> >
> > Jason
> >
>
>
>
> --
>
>
>
> --Brahma Reddy Battula
>


   

Re: I want to inquire about hadoop licence.

[Owen O'Malley]

There is a licensing FAQ for that 

http://www.apache.org/foundation/license-faq.html#IsItFee

.. Owen

> On Jan 17, 2018, at 11:06, 파란구름  wrote:
> 
> Dear in charge of the hadoop manager.
> 
> Thank you for providing hadoop service.
> I want to inquire about hadoop licence.
> Recently, I'm making a computer project to use hadoop service.
> I know hadoop is open source software.
> If my project is commercialized by using hadoop, then should i pay a fee?
> If yes, how much should i submit?
> 
> Thanks.

I want to inquire about hadoop licence.

[파란구름]

Dear in charge of the hadoop manager.

Thank you for providing hadoop service.
I want to inquire about hadoop licence.
Recently, I'm making a computer project to use hadoop service.
I know hadoop is open source software.
If my project is commercialized by using hadoop, then should i pay a fee?
If yes, how much should i submit?

Thanks.

Re: What's the difference between branch-3 and branch-3.0?

[Sangjin Lee]

+1 on deleting branch-3.

On Wed, Jan 17, 2018 at 10:42 AM, Jason Lowe  wrote:

> This was created accidentally when HDFS-11847 was committed.  As such we
> should delete the branch-3 branch and port over the commits that went into
> branch-3 instead of branch-3.0.  For the former, I'm assuming that requires
> an INFRA ticket since I would hope any committer would not have the ability
> to destroy a branch-* branch.  Unfortunately even after it's deleted I
> suspect we will see it reappear if someone pushes up their old copy of
> branch-3 again, so committers will need to be vigilant.
>
> I'll work on porting the missing changes below from branch-3 over to
> branch-3.0.  I'll wait for some more consensus on the branch-3 deletion
> before filing the INFRA ticket since deleting a branch shouldn't be done
> lightly.
>
> Jason
>
>
> commit 0802d8afa355d9a0683fdb2e9c4963e8fea8644f
> Author: Vinayakumar B 
> Date:   Wed Jan 17 14:16:48 2018 +0530
>
> HDFS-9049. Make Datanode Netty reverse proxy port to be configurable.
> Contributed by Vinayakumar B.
>
> (cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)
>
> commit db8345fa9cd124728d935f725525e2626438b4c1
> Author: Lei Xu 
> Date:   Tue Jan 16 15:15:11 2018 -0800
>
> HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when
> safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei)
>
> (cherry picked from commit 3bd9ea63df769345a9d02a404cfb61323a4cd7e3)
>
> commit 82741091a78d7ce62c240ec3e7f81a3a9a3fee36
> Author: Inigo Goiri 
> Date:   Mon Jan 15 12:21:24 2018 -0800
>
> HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer.
> Contributed by Inigo Goiri.
>
> commit d3fbcd92fe53192a319683b9ac72179cb28bd978
> Author: Yiqun Lin 
> Date:   Sat Jan 6 14:31:08 2018 +0800
>
> HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under
> a given path. Contributed by Yiqun Lin.
>
> commit ee44783515a55ab9fd368660c5cc2c2bc392132e
> Author: Manoj Govindassamy 
> Date:   Tue Jan 2 14:59:36 2018 -0800
>
> HDFS-11847. Enhance dfsadmin listOpenFiles command to list files
> blocking datanode decommissioning.
>
>
>
> On Wed, Jan 17, 2018 at 10:53 AM, Brahma Reddy Battula 
> wrote:
>
> > IMHO,we no need to have *branch-3* still trunk moves.Shall we remove as
> it
> > make confuses.??
> >
> >
> >
> >
> > Brahma Reddy Battula
> >
> > On Wed, Jan 17, 2018 at 9:41 PM, Jason Lowe 
> > wrote:
> >
> > > I recently noticed some committers posting commits to branch-3 and
> > marking
> > > the JIRA as fixed in 3.0.1.  I thought branch-3.0 was tracking 3.0.x
> > > releases, including branch-3.0.1 as of now, so I am confused what
> > branch-3
> > > is for.  The versions in the poms between branch-3 and branch-3.0 both
> > say
> > > they are 3.0.1-SNAPSHOT.
> > >
> > > I recall we discussed _not_ creating branch-3 until it is necessary,
> and
> > it
> > > is only necessary for branch-3 to exist when trunk stop tracking 3.x
> > > releases (i.e.: when trunk moves to 4.0.0-SNAPSHOT).
> > >
> > > Jason
> > >
> >
> >
> >
> > --
> >
> >
> >
> > --Brahma Reddy Battula
> >
>

Re: What's the difference between branch-3 and branch-3.0?

[Jason Lowe]

This was created accidentally when HDFS-11847 was committed.  As such we
should delete the branch-3 branch and port over the commits that went into
branch-3 instead of branch-3.0.  For the former, I'm assuming that requires
an INFRA ticket since I would hope any committer would not have the ability
to destroy a branch-* branch.  Unfortunately even after it's deleted I
suspect we will see it reappear if someone pushes up their old copy of
branch-3 again, so committers will need to be vigilant.

I'll work on porting the missing changes below from branch-3 over to
branch-3.0.  I'll wait for some more consensus on the branch-3 deletion
before filing the INFRA ticket since deleting a branch shouldn't be done
lightly.

Jason


commit 0802d8afa355d9a0683fdb2e9c4963e8fea8644f
Author: Vinayakumar B 
Date:   Wed Jan 17 14:16:48 2018 +0530

HDFS-9049. Make Datanode Netty reverse proxy port to be configurable.
Contributed by Vinayakumar B.

(cherry picked from commit 09efdfe9e13c9695867ce4034aa6ec970c2032f1)

commit db8345fa9cd124728d935f725525e2626438b4c1
Author: Lei Xu 
Date:   Tue Jan 16 15:15:11 2018 -0800

HDFS-13004. TestLeaseRecoveryStriped.testLeaseRecovery is failing when
safeLength is 0MB or larger than the test file. (Zsolt Venczel via lei)

(cherry picked from commit 3bd9ea63df769345a9d02a404cfb61323a4cd7e3)

commit 82741091a78d7ce62c240ec3e7f81a3a9a3fee36
Author: Inigo Goiri 
Date:   Mon Jan 15 12:21:24 2018 -0800

HDFS-12919. RBF: Support erasure coding methods in RouterRpcServer.
Contributed by Inigo Goiri.

commit d3fbcd92fe53192a319683b9ac72179cb28bd978
Author: Yiqun Lin 
Date:   Sat Jan 6 14:31:08 2018 +0800

HDFS-11848. Enhance dfsadmin listOpenFiles command to list files under
a given path. Contributed by Yiqun Lin.

commit ee44783515a55ab9fd368660c5cc2c2bc392132e
Author: Manoj Govindassamy 
Date:   Tue Jan 2 14:59:36 2018 -0800

HDFS-11847. Enhance dfsadmin listOpenFiles command to list files
blocking datanode decommissioning.



On Wed, Jan 17, 2018 at 10:53 AM, Brahma Reddy Battula 
wrote:

> IMHO,we no need to have *branch-3* still trunk moves.Shall we remove as it
> make confuses.??
>
>
>
>
> Brahma Reddy Battula
>
> On Wed, Jan 17, 2018 at 9:41 PM, Jason Lowe 
> wrote:
>
> > I recently noticed some committers posting commits to branch-3 and
> marking
> > the JIRA as fixed in 3.0.1.  I thought branch-3.0 was tracking 3.0.x
> > releases, including branch-3.0.1 as of now, so I am confused what
> branch-3
> > is for.  The versions in the poms between branch-3 and branch-3.0 both
> say
> > they are 3.0.1-SNAPSHOT.
> >
> > I recall we discussed _not_ creating branch-3 until it is necessary, and
> it
> > is only necessary for branch-3 to exist when trunk stop tracking 3.x
> > releases (i.e.: when trunk moves to 4.0.0-SNAPSHOT).
> >
> > Jason
> >
>
>
>
> --
>
>
>
> --Brahma Reddy Battula
>

Re: What's the difference between branch-3 and branch-3.0?

[Brahma Reddy Battula]

IMHO,we no need to have *branch-3* still trunk moves.Shall we remove as it
make confuses.??




Brahma Reddy Battula

On Wed, Jan 17, 2018 at 9:41 PM, Jason Lowe  wrote:

> I recently noticed some committers posting commits to branch-3 and marking
> the JIRA as fixed in 3.0.1.  I thought branch-3.0 was tracking 3.0.x
> releases, including branch-3.0.1 as of now, so I am confused what branch-3
> is for.  The versions in the poms between branch-3 and branch-3.0 both say
> they are 3.0.1-SNAPSHOT.
>
> I recall we discussed _not_ creating branch-3 until it is necessary, and it
> is only necessary for branch-3 to exist when trunk stop tracking 3.x
> releases (i.e.: when trunk moves to 4.0.0-SNAPSHOT).
>
> Jason
>



-- 



--Brahma Reddy Battula

[jira] [Created] (HADOOP-15177) Update the release year to 2018

[Akira Ajisaka (JIRA)]

Akira Ajisaka created HADOOP-15177:
--

 Summary: Update the release year to 2018
 Key: HADOOP-15177
 URL: https://issues.apache.org/jira/browse/HADOOP-15177
 Project: Hadoop Common
  Issue Type: Task
  Components: build
Reporter: Akira Ajisaka


The release year needs to be updated.
{noformat}
$ find . -name "pom.xml" | xargs grep -n 2017
./hadoop-project/pom.xml:34:    2017{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

What's the difference between branch-3 and branch-3.0?

[Jason Lowe]

I recently noticed some committers posting commits to branch-3 and marking
the JIRA as fixed in 3.0.1.  I thought branch-3.0 was tracking 3.0.x
releases, including branch-3.0.1 as of now, so I am confused what branch-3
is for.  The versions in the poms between branch-3 and branch-3.0 both say
they are 3.0.1-SNAPSHOT.

I recall we discussed _not_ creating branch-3 until it is necessary, and it
is only necessary for branch-3 to exist when trunk stop tracking 3.x
releases (i.e.: when trunk moves to 4.0.0-SNAPSHOT).

Jason

[jira] [Created] (HADOOP-15176) Enhance IAM assumed role support in S3A client

[Steve Loughran (JIRA)]

Steve Loughran created HADOOP-15176:
---

 Summary: Enhance IAM assumed role support in S3A client
 Key: HADOOP-15176
 URL: https://issues.apache.org/jira/browse/HADOOP-15176
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3, test
Affects Versions: 3.1.0
 Environment: Followup HADOOP-15141 with

* Code to generate basic AWS json policies somewhat declaratively (no hand 
coded strings)
* Tests to simulate users with different permissions down the path of a single 
bucket
* test-driven changes to S3A client to handle user without full write up the FS 
tree
* move the new authenticator into the s3a sub-package "auth", where we can put 
more auth stuff (that base s3a package is getting way too big)


Reporter: Steve Loughran
Assignee: Steve Loughran






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/107/

[Jan 16, 2018 3:31:02 AM] (rohithsharmaks) YARN-6736. Consider writing to both 
ats v1 & v2 from RM for smoother
[Jan 16, 2018 10:59:13 AM] (brahma) HDFS-8693. refreshNamenodes does not 
support adding a new standby to a




-1 overall


The following subsystems voted -1:
asflicense unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Unreaped Processes :

   hadoop-hdfs:42 
   bkjournal:8 
   hadoop-yarn-client:5 
   hadoop-yarn-applications-distributedshell:1 
   hadoop-mapreduce-client-jobclient:2 
   hadoop-distcp:3 

Failed junit tests :

   hadoop.hdfs.server.namenode.snapshot.TestSnapshottableDirListing 
   hadoop.hdfs.server.namenode.ha.TestPipelinesFailover 
   hadoop.hdfs.server.balancer.TestBalancer 
   hadoop.hdfs.server.namenode.TestQuotaByStorageType 
   hadoop.hdfs.server.namenode.ha.TestHAMetrics 
   hadoop.hdfs.server.blockmanagement.TestSlowDiskTracker 
   hadoop.hdfs.server.namenode.TestSecurityTokenEditLog 
   hadoop.hdfs.server.namenode.TestFileLimit 
   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits 
   hadoop.hdfs.server.namenode.snapshot.TestCheckpointsWithSnapshots 
   hadoop.hdfs.server.namenode.TestFSImageWithAcl 
   hadoop.hdfs.server.namenode.TestFavoredNodesEndToEnd 
   hadoop.hdfs.server.namenode.TestListOpenFiles 
   hadoop.hdfs.server.namenode.ha.TestEditLogsDuringFailover 
   hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup 
   hadoop.hdfs.server.namenode.TestStreamFile 
   hadoop.hdfs.server.blockmanagement.TestNameNodePrunesMissingStorages 
   hadoop.hdfs.server.namenode.snapshot.TestDisallowModifyROSnapshot 
   hadoop.hdfs.server.namenode.TestDecommissioningStatus 
   hadoop.hdfs.server.namenode.TestAuditLogger 
   hadoop.hdfs.server.namenode.TestGenericJournalConf 
   hadoop.hdfs.server.namenode.TestTransferFsImage 
   hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithQJM 
   hadoop.hdfs.server.mover.TestMover 
   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks 
   hadoop.hdfs.server.namenode.TestAclConfigFlag 
   hadoop.hdfs.server.namenode.ha.TestDelegationTokensWithHA 
   hadoop.hdfs.server.namenode.TestSecondaryWebUi 
   hadoop.hdfs.server.namenode.snapshot.TestNestedSnapshots 
   hadoop.hdfs.server.namenode.snapshot.TestSnapshotBlocksMap 
   hadoop.hdfs.server.namenode.ha.TestXAttrsWithHA 
   hadoop.hdfs.server.namenode.snapshot.TestXAttrWithSnapshot 
   hadoop.hdfs.server.namenode.web.resources.TestWebHdfsDataLocality 
   hadoop.hdfs.server.datanode.TestStorageReport 
   hadoop.hdfs.server.federation.router.TestNamenodeHeartbeat 
   hadoop.hdfs.server.namenode.TestCacheDirectives 
   hadoop.hdfs.server.namenode.TestProtectedDirectories 
   hadoop.hdfs.server.namenode.TestLargeDirectoryDelete 
   hadoop.hdfs.server.namenode.TestXAttrConfigFlag 
   hadoop.hdfs.server.blockmanagement.TestBlockStatsMXBean 
   hadoop.hdfs.server.namenode.snapshot.TestAclWithSnapshot 
   hadoop.hdfs.server.namenode.ha.TestDNFencingWithReplication 
   hadoop.hdfs.server.namenode.snapshot.TestSnapshot 
   hadoop.hdfs.server.namenode.TestBackupNode 
   hadoop.hdfs.server.balancer.TestBalancerWithSaslDataTransfer 
   hadoop.hdfs.server.namenode.ha.TestHarFileSystemWithHA 
   hadoop.hdfs.server.federation.router.TestRouterRpcMultiDestination 
   hadoop.hdfs.server.namenode.TestStartup 
   hadoop.hdfs.server.namenode.snapshot.TestFileContextSnapshot 
   
hadoop.hdfs.server.namenode.snapshot.TestSnapshotNameWithInvalidCharacters 
   hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFS 
   hadoop.hdfs.server.blockmanagement.TestNodeCount 
   hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA 
   hadoop.hdfs.server.namenode.ha.TestHAStateTransitions 
   hadoop.hdfs.server.namenode.TestSaveNamespace 
   hadoop.hdfs.server.namenode.TestNameNodeRpcServerMethods 
   hadoop.hdfs.server.namenode.snapshot.TestOpenFilesWithSnapshot 
   hadoop.hdfs.server.federation.store.driver.TestStateStoreFileSystem 
   hadoop.hdfs.server.namenode.TestDeadDatanode 
   hadoop.hdfs.server.namenode.ha.TestEditLogTailer 
   hadoop.hdfs.server.balancer.TestBalancerRPCDelay 
   hadoop.hdfs.server.namenode.ha.TestStateTransitionFailure 
   hadoop.hdfs.server.namenode.TestNameNodeResourceChecker 
   hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion 
   
hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain