date:20171218

[jira] [Created] (YARN-7673) ClassNotFoundException: org.apache.hadoop.yarn.server.api.DistributedSchedulingAMProtocol when using hadoop-client-minicluster

2017-12-18 Thread Jeff Zhang (JIRA)

Jeff Zhang created YARN-7673:


 Summary: ClassNotFoundException: 
org.apache.hadoop.yarn.server.api.DistributedSchedulingAMProtocol when using 
hadoop-client-minicluster
 Key: YARN-7673
 URL: https://issues.apache.org/jira/browse/YARN-7673
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Jeff Zhang


I'd like to use hadoop-client-minicluster for hadoop downstream project, but I 
encounter the following exception when starting hadoop minicluster.  And I 
check the hadoop-client-minicluster, it indeed does not have this class. Is 
this something that is missing when packaging the published jar ?

{code}
java.lang.NoClassDefFoundError: 
org/apache/hadoop/yarn/server/api/DistributedSchedulingAMProtocol

at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
at 
java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:467)
at java.net.URLClassLoader.access$100(URLClassLoader.java:73)
at java.net.URLClassLoader$1.run(URLClassLoader.java:368)
at java.net.URLClassLoader$1.run(URLClassLoader.java:362)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:361)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at 
org.apache.hadoop.yarn.server.MiniYARNCluster.createResourceManager(MiniYARNCluster.java:851)
at 
org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:285)
at 
org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
{code}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7672) hadoop-sls can not simulate huge scale of YARN

2017-12-18 Thread zhangshilong (JIRA)

zhangshilong created YARN-7672:
--

 Summary: hadoop-sls can not simulate huge scale of YARN
 Key: YARN-7672
 URL: https://issues.apache.org/jira/browse/YARN-7672
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: zhangshilong
Assignee: zhangshilong


Our YARN cluster scale to nearly 10 thousands nodes.
We need to do scheduler pressure test.
we start  2000+ threads to simulate NM and AM. So  cpu.load very high to 100+. 
I thought that will affect  performance evaluation of scheduler. 
So I thought to separate the scheduler from the simulator.
I start a real RM. Then SLS will register nodes to RM,And submit apps to RM 
using RM RPC.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[ANNOUNCE] Apache Hadoop 2.8.3 Release

2017-12-18 Thread 俊平堵

Hi all,

I am pleased to announce that the Apache Hadoop community has voted to
release Apache Hadoop 2.8.3!

Apache Hadoop 2.8.3 is the next GA release of Apache Hadoop 2.8 line, which
include
79 fixes identified for previous Hadoop 2.8.2 releases:
 - For major changes included in Hadoop 2.8 line, please refer Hadoop 2.8.3
main page [1].
 - For more details about fixes in 2.8.3 release, please read the log of
CHANGES [2] and RELEASENOTES [3].

The release news is posted on the Hadoop website too, you can go to the
downloads section directly [4].

Thank you all for contributing to the Apache Hadoop!


Cheers,


Junping

[1] http://hadoop.apache.org/docs/r2.8.3/index.html

[2]
http://hadoop.apache.org/docs/r2.8.3/hadoop-project-dist/hadoop-common/release/2.8.3/CHANGES.2.8.3.html

[3]
http://hadoop.apache.org/docs/r2.8.3/hadoop-project-dist/hadoop-common/release/2.8.3/RELEASENOTES.2.8.3.html

[4] http://hadoop.apache.org/releases.html#Download

[jira] [Created] (YARN-7671) Improve Diagonstic message for stop yarn native service

2017-12-18 Thread Yesha Vora (JIRA)

Yesha Vora created YARN-7671:


 Summary: Improve Diagonstic message for stop yarn native service 
 Key: YARN-7671
 URL: https://issues.apache.org/jira/browse/YARN-7671
 Project: Hadoop YARN
  Issue Type: Bug
Affects Versions: 3.0.0
Reporter: Yesha Vora


Steps:

1) Install Hadoop 3.0 cluster
2) Run Yarn service application
{code:title=sleeper.json}{
  "name": "sleeper-service",
  "components" : 
[
  {
"name": "sleeper",
"number_of_containers": 1,
"launch_command": "sleep 90",
"resource": {
  "cpus": 1, 
  "memory": "256"
   }
  }
]
}{code}
{code:title=cmd}
yarn app -launch my-sleeper1 sleeper.json{code}
3) stop yarn service app 
{code:title=cmd}
yarn app -stop my-sleeper1{code}

On stopping yarn service, appId finishes with YarnApplicationState: FINISHED , 
FinalStatus Reported by AM: ENDED and Diagnostics:   Navigate to the failed 
component for more details.

Here,  Diagnostics message should be improved. When an application is 
explicitly stopped by user, the diagnostics message should say " Application 
stopped by user" 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

2017-12-18 Thread Andrew Wang

Thanks for the spot, I just pushed a correct tag. I can't delete the bad
tag myself, will ask ASF infra for help.

On Mon, Dec 18, 2017 at 4:46 PM, Jonathan Kelly 
wrote:

> Congrats on the huge release!
>
> I just noticed, though, that the Github repo does not appear to have the
> correct tag for 3.0.0. I see a new tag called "rel/release-" that points to
> the same commit as "release-3.0.0-RC1" 
> (c25427ceca461ee979d30edd7a4b0f50718e6533).
> I assume that should have actually been called "rel/release-3.0.0" to match
> the pattern for prior releases.
>
> Thanks,
> Jonathan Kelly
>
> On Thu, Dec 14, 2017 at 10:45 AM Andrew Wang 
> wrote:
>
>> Hi all,
>>
>> I'm pleased to announce that Apache Hadoop 3.0.0 is generally available
>> (GA).
>>
>> 3.0.0 GA consists of 302 bug fixes, improvements, and other enhancements
>> since 3.0.0-beta1. This release marks a point of quality and stability for
>> the 3.0.0 release line, and users of earlier 3.0.0-alpha and -beta
>> releases
>> are encouraged to upgrade.
>>
>> Looking back, 3.0.0 GA is the culmination of over a year of work on the
>> 3.0.0 line, starting with 3.0.0-alpha1 which was released in September
>> 2016. Altogether, 3.0.0 incorporates 6,242 changes since 2.7.0.
>>
>> Users are encouraged to read the overview of major changes
>>  in 3.0.0. The GA
>> release
>> notes
>> > dist/hadoop-common/release/3.0.0/RELEASENOTES.3.0.0.html>
>>  and changelog
>> > dist/hadoop-common/release/3.0.0/CHANGES.3.0.0.html>
>> detail
>> the changes since 3.0.0-beta1.
>>
>> The ASF press release provides additional color and highlights some of the
>> major features:
>>
>> https://globenewswire.com/news-release/2017/12/14/
>> 1261879/0/en/The-Apache-Software-Foundation-Announces-
>> Apache-Hadoop-v3-0-0-General-Availability.html
>>
>> Let me end by thanking the many, many contributors who helped with this
>> release line. We've only had three major releases in Hadoop's 10 year
>> history, and this is our biggest major release ever. It's an incredible
>> accomplishment for our community, and I'm proud to have worked with all of
>> you.
>>
>> Best,
>> Andrew
>>
>

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

2017-12-18 Thread Jonathan Kelly

Congrats on the huge release!

I just noticed, though, that the Github repo does not appear to have the
correct tag for 3.0.0. I see a new tag called "rel/release-" that points to
the same commit as "release-3.0.0-RC1"
(c25427ceca461ee979d30edd7a4b0f50718e6533). I assume that should have
actually been called "rel/release-3.0.0" to match the pattern for prior
releases.

Thanks,
Jonathan Kelly

On Thu, Dec 14, 2017 at 10:45 AM Andrew Wang 
wrote:

> Hi all,
>
> I'm pleased to announce that Apache Hadoop 3.0.0 is generally available
> (GA).
>
> 3.0.0 GA consists of 302 bug fixes, improvements, and other enhancements
> since 3.0.0-beta1. This release marks a point of quality and stability for
> the 3.0.0 release line, and users of earlier 3.0.0-alpha and -beta releases
> are encouraged to upgrade.
>
> Looking back, 3.0.0 GA is the culmination of over a year of work on the
> 3.0.0 line, starting with 3.0.0-alpha1 which was released in September
> 2016. Altogether, 3.0.0 incorporates 6,242 changes since 2.7.0.
>
> Users are encouraged to read the overview of major changes
>  in 3.0.0. The GA release
> notes
> <
> http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/release/3.0.0/RELEASENOTES.3.0.0.html
> >
>  and changelog
> <
> http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/release/3.0.0/CHANGES.3.0.0.html
> >
> detail
> the changes since 3.0.0-beta1.
>
> The ASF press release provides additional color and highlights some of the
> major features:
>
>
> https://globenewswire.com/news-release/2017/12/14/1261879/0/en/The-Apache-Software-Foundation-Announces-Apache-Hadoop-v3-0-0-General-Availability.html
>
> Let me end by thanking the many, many contributors who helped with this
> release line. We've only had three major releases in Hadoop's 10 year
> history, and this is our biggest major release ever. It's an incredible
> accomplishment for our community, and I'm proud to have worked with all of
> you.
>
> Best,
> Andrew
>

[jira] [Created] (YARN-7670) Modifications to the ResourceScheduler to support SchedulingRequests

2017-12-18 Thread Arun Suresh (JIRA)

Arun Suresh created YARN-7670:
-

 Summary: Modifications to the ResourceScheduler to support 
SchedulingRequests
 Key: YARN-7670
 URL: https://issues.apache.org/jira/browse/YARN-7670
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Arun Suresh
Assignee: Arun Suresh


As per discussions in YARN-7612. This JIRA tracks the changes to the 
ResourceScheduler interface and implementation in CapacityScheduler to support 
SchedulingRequests



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7669) [API] Introduce interfaces for placement constraint processing

2017-12-18 Thread Arun Suresh (JIRA)

Arun Suresh created YARN-7669:
-

 Summary: [API] Introduce interfaces for placement constraint 
processing
 Key: YARN-7669
 URL: https://issues.apache.org/jira/browse/YARN-7669
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Arun Suresh
Assignee: Arun Suresh


As per discussions in YARN-7612. This JIRA will introduce the generic 
interfaces which will be implemented in YARN-7612



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7668) Remove unused variables from ContainerLocalizer

2017-12-18 Thread Ray Chiang (JIRA)

Ray Chiang created YARN-7668:


 Summary: Remove unused variables from ContainerLocalizer
 Key: YARN-7668
 URL: https://issues.apache.org/jira/browse/YARN-7668
 Project: Hadoop YARN
  Issue Type: Task
Reporter: Ray Chiang
Assignee: Ray Chiang
Priority: Trivial


While figuring out something else, I found two class constants in 
ContainerLocalizer that look like aren't being used anymore.

{noformat}
  public static final String OUTPUTDIR = "output";
  public static final String WORKDIR = "work";
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7667) Docker Stop grace period should be configurable

2017-12-18 Thread Eric Badger (JIRA)

Eric Badger created YARN-7667:
-

 Summary: Docker Stop grace period should be configurable
 Key: YARN-7667
 URL: https://issues.apache.org/jira/browse/YARN-7667
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Eric Badger


{{DockerStopCommand}} has a {{setGracePeriod}} method, but it is never called. 
So, the stop uses the 10 second default grace period from docker



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

2017-12-18 Thread Allen Wittenauer


It’s significantly more concerning that 3.0.0-beta1 doesn’t show up here:

http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/release/index.html

It looks like they are missing from the source tag too.  I wonder what else is 
missing.


> On Dec 18, 2017, at 11:15 AM, Andrew Wang  wrote:
> 
> Moving general@ to BCC,
> 
> The main page and releases posts on hadoop.apache.org are pretty clear
> about this being a diff from beta1, am I missing something? Pasted below:
> 
> After four alpha releases and one beta release, 3.0.0 is generally
> available. 3.0.0 consists of 302 bug fixes, improvements, and other
> enhancements since 3.0.0-beta1. All together, 6242 issues were fixed as
> part of the 3.0.0 release series since 2.7.0.
> 
> Users are encouraged to read the overview of major changes
>  in 3.0.0. The GA release
> notes
> 
> and changelog
> 
> detail
> the changes since 3.0.0-beta1.
> 
> 
> 
> On Mon, Dec 18, 2017 at 10:32 AM, Arpit Agarwal 
> wrote:
> 
>> That makes sense for Beta users but most of our users will be upgrading
>> from a previous GA release and the changelog will mislead them. The webpage
>> does not mention this is a delta from the beta release.
>> 
>> 
>> 
>> 
>> 
>> *From: *Andrew Wang 
>> *Date: *Friday, December 15, 2017 at 10:36 AM
>> *To: *Arpit Agarwal 
>> *Cc: *general , "common-...@hadoop.apache.org"
>> , "yarn-dev@hadoop.apache.org" <
>> yarn-dev@hadoop.apache.org>, "mapreduce-...@hadoop.apache.org" <
>> mapreduce-...@hadoop.apache.org>, "hdfs-...@hadoop.apache.org" <
>> hdfs-...@hadoop.apache.org>
>> *Subject: *Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released
>> 
>> 
>> 
>> Hi Arpit,
>> 
>> 
>> 
>> If you look at the release announcements, it's made clear that the
>> changelog for 3.0.0 is diffed based on beta1. This is important since users
>> need to know what's different from the previous 3.0.0-* releases if they're
>> upgrading.
>> 
>> 
>> 
>> I agree there's additional value to making combined release notes, but
>> it'd be something additive rather than replacing what's there.
>> 
>> 
>> 
>> Best,
>> 
>> Andrew
>> 
>> 
>> 
>> On Fri, Dec 15, 2017 at 8:27 AM, Arpit Agarwal 
>> wrote:
>> 
>> 
>> Hi Andrew,
>> 
>> Thank you for all the hard work on this release. I was out the last few
>> days and didn’t get a chance to evaluate RC1 earlier.
>> 
>> The changelog looks incorrect. E.g. This gives an impression that there
>> are just 5 incompatible changes in 3.0.0.
>> http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/
>> hadoop-common/release/3.0.0/CHANGES.3.0.0.html
>> 
>> I assume you only counted 3.0.0 changes in this log excluding
>> alphas/betas. However, users shouldn’t have to manually compile
>> incompatibilities by summing up a/b release notes. Can we fix the changelog
>> after the fact?
>> 
>> 
>> 
>> 
>> On 12/14/17, 10:45 AM, "Andrew Wang"  wrote:
>> 
>>Hi all,
>> 
>>I'm pleased to announce that Apache Hadoop 3.0.0 is generally available
>>(GA).
>> 
>>3.0.0 GA consists of 302 bug fixes, improvements, and other
>> enhancements
>>since 3.0.0-beta1. This release marks a point of quality and stability
>> for
>>the 3.0.0 release line, and users of earlier 3.0.0-alpha and -beta
>> releases
>>are encouraged to upgrade.
>> 
>>Looking back, 3.0.0 GA is the culmination of over a year of work on the
>>3.0.0 line, starting with 3.0.0-alpha1 which was released in September
>>2016. Altogether, 3.0.0 incorporates 6,242 changes since 2.7.0.
>> 
>>Users are encouraged to read the overview of major changes
>> in 3.0.0. The GA
>> release
>>notes
>>> dist/hadoop-common/release/3.0.0/RELEASENOTES.3.0.0.html>
>> and changelog
>>> dist/hadoop-common/release/3.0.0/CHANGES.3.0.0.html>
>> 
>>detail
>>the changes since 3.0.0-beta1.
>> 
>>The ASF press release provides additional color and highlights some of
>> the
>>major features:
>> 
>>https://globenewswire.com/news-release/2017/12/14/
>> 1261879/0/en/The-Apache-Software-Foundation-Announces-
>> Apache-Hadoop-v3-0-0-General-Availability.html
>> 
>>Let me end by thanking the many, many contributors who helped with this
>>release line. We've only had three major releases in Hadoop's 10 year
>>history, and this is our biggest major release ever. It's an incredible
>>accomplishment for our

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

2017-12-18 Thread Andrew Wang

Moving general@ to BCC,

The main page and releases posts on hadoop.apache.org are pretty clear
about this being a diff from beta1, am I missing something? Pasted below:

After four alpha releases and one beta release, 3.0.0 is generally
available. 3.0.0 consists of 302 bug fixes, improvements, and other
enhancements since 3.0.0-beta1. All together, 6242 issues were fixed as
part of the 3.0.0 release series since 2.7.0.

Users are encouraged to read the overview of major changes
 in 3.0.0. The GA release
notes

 and changelog

detail
the changes since 3.0.0-beta1.



On Mon, Dec 18, 2017 at 10:32 AM, Arpit Agarwal 
wrote:

> That makes sense for Beta users but most of our users will be upgrading
> from a previous GA release and the changelog will mislead them. The webpage
> does not mention this is a delta from the beta release.
>
>
>
>
>
> *From: *Andrew Wang 
> *Date: *Friday, December 15, 2017 at 10:36 AM
> *To: *Arpit Agarwal 
> *Cc: *general , "common-...@hadoop.apache.org"
> , "yarn-dev@hadoop.apache.org" <
> yarn-dev@hadoop.apache.org>, "mapreduce-...@hadoop.apache.org" <
> mapreduce-...@hadoop.apache.org>, "hdfs-...@hadoop.apache.org" <
> hdfs-...@hadoop.apache.org>
> *Subject: *Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released
>
>
>
> Hi Arpit,
>
>
>
> If you look at the release announcements, it's made clear that the
> changelog for 3.0.0 is diffed based on beta1. This is important since users
> need to know what's different from the previous 3.0.0-* releases if they're
> upgrading.
>
>
>
> I agree there's additional value to making combined release notes, but
> it'd be something additive rather than replacing what's there.
>
>
>
> Best,
>
> Andrew
>
>
>
> On Fri, Dec 15, 2017 at 8:27 AM, Arpit Agarwal 
> wrote:
>
>
> Hi Andrew,
>
> Thank you for all the hard work on this release. I was out the last few
> days and didn’t get a chance to evaluate RC1 earlier.
>
> The changelog looks incorrect. E.g. This gives an impression that there
> are just 5 incompatible changes in 3.0.0.
> http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/
> hadoop-common/release/3.0.0/CHANGES.3.0.0.html
>
> I assume you only counted 3.0.0 changes in this log excluding
> alphas/betas. However, users shouldn’t have to manually compile
> incompatibilities by summing up a/b release notes. Can we fix the changelog
> after the fact?
>
>
>
>
> On 12/14/17, 10:45 AM, "Andrew Wang"  wrote:
>
> Hi all,
>
> I'm pleased to announce that Apache Hadoop 3.0.0 is generally available
> (GA).
>
> 3.0.0 GA consists of 302 bug fixes, improvements, and other
> enhancements
> since 3.0.0-beta1. This release marks a point of quality and stability
> for
> the 3.0.0 release line, and users of earlier 3.0.0-alpha and -beta
> releases
> are encouraged to upgrade.
>
> Looking back, 3.0.0 GA is the culmination of over a year of work on the
> 3.0.0 line, starting with 3.0.0-alpha1 which was released in September
> 2016. Altogether, 3.0.0 incorporates 6,242 changes since 2.7.0.
>
> Users are encouraged to read the overview of major changes
>  in 3.0.0. The GA
> release
> notes
>  dist/hadoop-common/release/3.0.0/RELEASENOTES.3.0.0.html>
>  and changelog
>  dist/hadoop-common/release/3.0.0/CHANGES.3.0.0.html>
>
> detail
> the changes since 3.0.0-beta1.
>
> The ASF press release provides additional color and highlights some of
> the
> major features:
>
> https://globenewswire.com/news-release/2017/12/14/
> 1261879/0/en/The-Apache-Software-Foundation-Announces-
> Apache-Hadoop-v3-0-0-General-Availability.html
>
> Let me end by thanking the many, many contributors who helped with this
> release line. We've only had three major releases in Hadoop's 10 year
> history, and this is our biggest major release ever. It's an incredible
> accomplishment for our community, and I'm proud to have worked with
> all of
> you.
>
> Best,
> Andrew
>
>
>
>
>
>
>
>

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

2017-12-18 Thread Arpit Agarwal

That makes sense for Beta users but most of our users will be upgrading from a 
previous GA release and the changelog will mislead them. The webpage does not 
mention this is a delta from the beta release.

From: Andrew Wang 
Date: Friday, December 15, 2017 at 10:36 AM
To: Arpit Agarwal 
Cc: general , "common-...@hadoop.apache.org" 
, "yarn-dev@hadoop.apache.org" 
, "mapreduce-...@hadoop.apache.org" 
, "hdfs-...@hadoop.apache.org" 

Subject: Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

Hi Arpit,

If you look at the release announcements, it's made clear that the changelog 
for 3.0.0 is diffed based on beta1. This is important since users need to know 
what's different from the previous 3.0.0-* releases if they're upgrading.

I agree there's additional value to making combined release notes, but it'd be 
something additive rather than replacing what's there.

Best,
Andrew

On Fri, Dec 15, 2017 at 8:27 AM, Arpit Agarwal 
> wrote:

Hi Andrew,

Thank you for all the hard work on this release. I was out the last few days 
and didn’t get a chance to evaluate RC1 earlier.

The changelog looks incorrect. E.g. This gives an impression that there are 
just 5 incompatible changes in 3.0.0.
http://hadoop.apache.org/docs/r3.0.0/hadoop-project-dist/hadoop-common/release/3.0.0/CHANGES.3.0.0.html

I assume you only counted 3.0.0 changes in this log excluding alphas/betas. 
However, users shouldn’t have to manually compile incompatibilities by summing 
up a/b release notes. Can we fix the changelog after the fact?

On 12/14/17, 10:45 AM, "Andrew Wang" 
> wrote:

Hi all,

I'm pleased to announce that Apache Hadoop 3.0.0 is generally available
(GA).

3.0.0 GA consists of 302 bug fixes, improvements, and other enhancements
since 3.0.0-beta1. This release marks a point of quality and stability for
the 3.0.0 release line, and users of earlier 3.0.0-alpha and -beta releases
are encouraged to upgrade.

Looking back, 3.0.0 GA is the culmination of over a year of work on the
3.0.0 line, starting with 3.0.0-alpha1 which was released in September
2016. Altogether, 3.0.0 incorporates 6,242 changes since 2.7.0.

Users are encouraged to read the overview of major changes
 in 3.0.0. The GA release
notes

 and changelog

detail
the changes since 3.0.0-beta1.

The ASF press release provides additional color and highlights some of the
major features:

https://globenewswire.com/news-release/2017/12/14/1261879/0/en/The-Apache-Software-Foundation-Announces-Apache-Hadoop-v3-0-0-General-Availability.html

Let me end by thanking the many, many contributors who helped with this
release line. We've only had three major releases in Hadoop's 10 year
history, and this is our biggest major release ever. It's an incredible
accomplishment for our community, and I'm proud to have worked with all of
you.

Best,
Andrew

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-12-18 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/626/

[Dec 18, 2017 2:07:16 AM] (wwei) YARN-7617. Add a flag in distributed shell to 
automatically PROMOTE
[Dec 18, 2017 1:24:51 PM] (aajisaka) YARN-7664. Several javadoc errors. 
Contributed by Sean Mackrory.

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2017-12-18 Thread Apache Jenkins Server

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/72/

No changes




-1 overall


The following subsystems voted -1:
asflicense unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Unreaped Processes :

   hadoop-common:1 
   hadoop-hdfs:20 
   bkjournal:5 
   hadoop-yarn-server-timelineservice:1 
   hadoop-yarn-client:4 
   hadoop-yarn-applications-distributedshell:1 
   hadoop-mapreduce-client-jobclient:12 
   hadoop-distcp:5 
   hadoop-archives:1 
   hadoop-extras:1 

Failed junit tests :

   hadoop.fs.sftp.TestSFTPFileSystem 
   
hadoop.yarn.server.nodemanager.containermanager.scheduler.TestContainerSchedulerQueuing
 
   
hadoop.yarn.server.resourcemanager.reservation.TestCapacityOverTimePolicy 
   
hadoop.yarn.applications.distributedshell.TestDistributedShellWithNodeLabels 
   hadoop.mapreduce.lib.input.TestMultipleInputs 
   hadoop.mapreduce.lib.output.TestJobOutputCommitter 
   hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedComparator 
   hadoop.mapreduce.v2.TestMRAMWithNonNormalizedCapabilities 
   hadoop.mapreduce.lib.input.TestMRSequenceFileInputFilter 
   hadoop.mapreduce.lib.input.TestNLineInputFormat 
   hadoop.mapreduce.v2.TestNonExistentJob 
   hadoop.mapreduce.lib.input.TestMRSequenceFileAsBinaryInputFormat 
   hadoop.mapreduce.v2.TestMRAppWithCombiner 
   hadoop.mapreduce.TestLocalRunner 
   hadoop.mapreduce.v2.TestUberAM 
   hadoop.mapreduce.v2.TestMRJobsWithProfiler 
   hadoop.mapreduce.lib.output.TestMRMultipleOutputs 
   hadoop.mapreduce.TestMapperReducerCleanup 
   hadoop.mapreduce.v2.TestRMNMInfo 
   hadoop.mapreduce.v2.TestSpeculativeExecutionWithMRApp 
   hadoop.mapreduce.v2.TestSpeculativeExecution 
   hadoop.mapreduce.v2.TestMROldApiJobs 
   hadoop.mapreduce.TestValueIterReset 
   hadoop.mapreduce.lib.input.TestLineRecordReaderJobs 
   hadoop.mapreduce.v2.TestMRJobsWithHistoryService 
   hadoop.mapreduce.lib.chain.TestChainErrors 
   hadoop.mapreduce.lib.input.TestMRCJCFileInputFormat 
   hadoop.mapreduce.lib.fieldsel.TestMRFieldSelection 
   hadoop.mapreduce.lib.output.TestMRSequenceFileAsBinaryOutputFormat 
   hadoop.mapreduce.lib.aggregate.TestMapReduceAggregates 
   hadoop.mapreduce.TestLargeSort 
   hadoop.mapreduce.lib.input.TestMRSequenceFileAsTextInputFormat 
   hadoop.mapreduce.lib.chain.TestMapReduceChain 
   hadoop.mapreduce.lib.db.TestDataDrivenDBInputFormat 
   hadoop.mapreduce.lib.output.TestMRCJCFileOutputCommitter 
   hadoop.mapreduce.lib.chain.TestSingleElementChain 
   hadoop.tools.TestDistCpSystem 
   hadoop.tools.TestIntegration 
   hadoop.tools.TestDistCpViewFs 
   hadoop.resourceestimator.solver.impl.TestLpSolver 
   hadoop.resourceestimator.service.TestResourceEstimatorService 

Timed out junit tests :

   org.apache.hadoop.log.TestLogLevel 
   org.apache.hadoop.hdfs.TestLeaseRecovery2 
   org.apache.hadoop.hdfs.TestDatanodeRegistration 
   org.apache.hadoop.hdfs.TestRead 
   org.apache.hadoop.hdfs.web.TestWebHdfsTokens 
   org.apache.hadoop.hdfs.TestDFSInotifyEventInputStream 
   org.apache.hadoop.hdfs.TestDatanodeLayoutUpgrade 
   org.apache.hadoop.hdfs.TestFileAppendRestart 
   org.apache.hadoop.hdfs.security.TestDelegationToken 
   org.apache.hadoop.hdfs.web.TestWebHdfsWithRestCsrfPreventionFilter 
   org.apache.hadoop.hdfs.TestDFSOutputStream 
   org.apache.hadoop.hdfs.TestDatanodeReport 
   org.apache.hadoop.hdfs.web.TestWebHDFS 
   org.apache.hadoop.hdfs.web.TestWebHDFSXAttr 
   org.apache.hadoop.hdfs.web.TestWebHdfsWithMultipleNameNodes 
   org.apache.hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs 
   org.apache.hadoop.hdfs.TestDistributedFileSystem 
   org.apache.hadoop.hdfs.web.TestWebHDFSForHA 
   org.apache.hadoop.hdfs.TestReplaceDatanodeFailureReplication 
   org.apache.hadoop.hdfs.TestDFSShell 
   org.apache.hadoop.hdfs.web.TestWebHDFSAcl 
   org.apache.hadoop.contrib.bkjournal.TestBootstrapStandbyWithBKJM 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperJournalManager 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperAsHASharedDir 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperSpeculativeRead 
   
org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServices
 
   org.apache.hadoop.yarn.client.TestRMFailover 
   org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA

[jira] [Created] (YARN-7666) Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations

2017-12-18 Thread Sunil G (JIRA)

Sunil G created YARN-7666:
-

 Summary: Introduce scheduler specific environment variable support 
in ASC for better scheduling placement configurations
 Key: YARN-7666
 URL: https://issues.apache.org/jira/browse/YARN-7666
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Sunil G
Assignee: Sunil G


Introduce a scheduler specific key-value map to hold env variables in ASC.

And also convert AppPlacementAllocator initialization to each app based on 
policy configured at each app.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-7673) ClassNotFoundException: org.apache.hadoop.yarn.server.api.DistributedSchedulingAMProtocol when using hadoop-client-minicluster

[jira] [Created] (YARN-7672) hadoop-sls can not simulate huge scale of YARN

[ANNOUNCE] Apache Hadoop 2.8.3 Release

[jira] [Created] (YARN-7671) Improve Diagonstic message for stop yarn native service

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

[jira] [Created] (YARN-7670) Modifications to the ResourceScheduler to support SchedulingRequests

[jira] [Created] (YARN-7669) [API] Introduce interfaces for placement constraint processing

[jira] [Created] (YARN-7668) Remove unused variables from ContainerLocalizer

[jira] [Created] (YARN-7667) Docker Stop grace period should be configurable

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

Re: [ANNOUNCE] Apache Hadoop 3.0.0 GA is released

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[jira] [Created] (YARN-7666) Introduce scheduler specific environment variable support in ASC for better scheduling placement configurations

16 matches

Site Navigation

Mail list logo

Footer information