[jira] [Created] (YARN-7527) Over-allocate node resource in async-scheduling mode of CapacityScheduler

Tao Yang created YARN-7527: -- Summary: Over-allocate node resource in async-scheduling mode of CapacityScheduler Key: YARN-7527 URL: https://issues.apache.org/jira/browse/YARN-7527 Project: Hadoop YARN

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/43/ [Nov 15, 2017 6:33:48 PM] (jlowe) YARN-7361. Improve the docker container runtime documentation. [Nov 16, 2017 12:45:06 AM] (xiao) HADOOP-15023. ValueQueue should also validate (int) (lowWatermark *

[jira] [Created] (YARN-7528) Need to document that resource types that use units need to be defined at RM level and NM level

Grant Sohn created YARN-7528: Summary: Need to document that resource types that use units need to be defined at RM level and NM level Key: YARN-7528 URL: https://issues.apache.org/jira/browse/YARN-7528

[jira] [Created] (YARN-7529) TestYarnNativeServices#testRecoverComponentsAfterRMRestart() fails intermittently

Chandni Singh created YARN-7529: --- Summary: TestYarnNativeServices#testRecoverComponentsAfterRMRestart() fails intermittently Key: YARN-7529 URL: https://issues.apache.org/jira/browse/YARN-7529

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

+1 (binding) * Built from source * Set up a 4 node cluster with RM HA enabled * Performed some basic HDFS commands * Ran a bunch of pi & sleep jobs - with and without opportunistic containers * Performed some basic RM failover testing. Cheers -Arun On Thu, Nov 16, 2017 at 11:12 PM, Akira

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

Thanks for putting this release together! +1 (binding) - Verified signatures and digests - Successfully built from source including native - Deployed to single-node cluster and ran some test jobs Jason On Mon, Nov 13, 2017 at 6:10 PM, Arun Suresh wrote: > Hi Folks, > >

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Thanks for the spot, normally create-release spits those out. I uploaded asc and mds for the release artifacts. Best, Andrew On Thu, Nov 16, 2017 at 11:33 PM, Akira Ajisaka wrote: > Hi Andrew, > > Signatures are missing. Would you upload them? > > Thanks, > Akira > > > On

Re: [DISCUSS] Apache YARN committers/contribu­t­ors meetup #5

I will close the poll this afternoon.  Right now the overwhelming preference is for Dec 6th.  Remote folks note that we will have a dial-in option, most likely via Hangout. Daniel On 11/14/17 10:39 AM, Daniel Templeton wrote: I was hoping for some signs of life before posting a Doodle poll,

Re: [DISCUSS] Apache Hadoop 2.7.5 Release Plan

Hi Konstantin, Thanks for initiating the release effort. I am marking HDFS-12641 as a blocker for Hadoop 2.7.5 because during our internal testing for CDH we found a regression in HDFS-11445 that was fixed by HDFS-11755 (technically not a real

Re: Changing the JSON Serializer

+1 on changing the JSON serializer. Hadoop was an early adopter for Jersey, but proper JSON deserializer for Jackson didn’t appear until mid 2016 after Jackson 2.5 release. Hence, some early versions of Hadoop REST API were not JSON compliant. Hadoop kind of comply to schematic versioning,

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

Wrapping up the vote with my +1. Deployed RC3 on a federated YARN cluster with 6 sub-clusters: - ran multiple sample jobs - enabled opportunistic containers and submitted more samples - configured HDFS federation and reran jobs. With 13 binding +1s and 7 non-binding +1s and no +/-1s, pleased to

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi Andrew, Thank you for your hard work in getting us to this step. This is our first major GA release in many years. I feel a 5-day vote window ending over the weekend before thanksgiving may not provide sufficient time to evaluate this RC especially for downstream components. Would you

[jira] [Created] (YARN-7530) hadoop-yarn-services-api should be part of hadoop-yarn-services

Eric Yang created YARN-7530: --- Summary: hadoop-yarn-services-api should be part of hadoop-yarn-services Key: YARN-7530 URL: https://issues.apache.org/jira/browse/YARN-7530 Project: Hadoop YARN

Re: [DISCUSS] Apache YARN committers/contribu­t­ors meetup #5

Dec 6th remains the overwhelming preference, so let's go with it. I agree that we need to do a bug bash, but since it's been a while since we last had a meetup, I would say let's use this one to get the ball rolling again, and we can add an agenda item to plan for a bug bash.  Does that work?

Re: Changing the JSON Serializer

Yeah, we're not going to be able to change the REST APIs without updating the version number and leaving the old version around for a while.  We should make sure that the fix makes future WS version revs easy (or at least easier than this one). Daniel On 11/17/17 3:56 PM, Eric Yang wrote:

Re: Changing the JSON Serializer

I personally wouldn't be comfortable adding an API version in a maintenance release; it's essentially adding a feature. but I'm not set to be the RM for 3.0.1. :) On Nov 17, 2017 17:56, "Eric Yang" wrote: > This means YARN-7505 can have /ws/v2/* running in parallel of

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/44/ [Nov 16, 2017 5:12:52 PM] (sunilg) YARN-7469. Capacity Scheduler Intra-queue preemption: User can starve if [Nov 16, 2017 7:12:01 PM] (rkanter) HADOOP-14982. Clients using FailoverOnNetworkExceptionRetry

Re: [DISCUSS] Apache YARN committers/contribu­t­ors meetup #5

Thanks for volunteering to host, Daniel! The agenda looks good. But we could also go a completely orthogonal way and do a bug / review bash too. Up for either of these directions. Or a mix. Thanks +Vinod > On Nov 17, 2017, at 9:22 AM, Daniel Templeton wrote: > > I will

Re: Changing the JSON Serializer

Oh, yeah, I agree.  Adding the new version would be a 3.1 thing. Daniel On 11/17/17 4:48 PM, Sean Busbey wrote: I personally wouldn't be comfortable adding an API version in a maintenance release; it's essentially adding a feature. but I'm not set to be the RM for 3.0.1. :) On Nov 17, 2017

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

Hi Arpit, I agree the timing is not great here, but extending it to meaningfully avoid the holidays would mean extending it an extra week (e.g. to the 29th). We've been coordinating with ASF PR for that Tuesday, so I'd really, really like to get the RC out before then. In terms of downstream

Re: Changing the JSON Serializer

This means YARN-7505 can have /ws/v2/* running in parallel of /ws/v1/* for 3.1 or 3.0.1 release, and deprecate /ws/v1/*. In version 4, we drop /ws/v1/*, right? I think this plan can work. Regards, Eric From: Sean Busbey Date: Friday, November 17, 2017 at 3:08 PM To:

[jira] [Created] (YARN-7531) ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType()

Haibo Chen created YARN-7531: Summary: ResourceRequest.equal does not check ExecutionTypeRequest.enforceExecutionType() Key: YARN-7531 URL: https://issues.apache.org/jira/browse/YARN-7531 Project: Hadoop

Re: Changing the JSON Serializer

3.0.0 RCs are in progress already. Bit late to make a breaking change. the REST APIs are versioned for a reason. So long as we're outputting these changes on a new version, this change should be fine on whatever branch we like. When we open up for changes to go in the next major release we can

[jira] [Created] (YARN-7532) NM rolling restart causing application exceed max attempt

Bill Liu created YARN-7532: -- Summary: NM rolling restart causing application exceed max attempt Key: YARN-7532 URL: https://issues.apache.org/jira/browse/YARN-7532 Project: Hadoop YARN Issue Type:

Re: [DISCUSS] Apache YARN committers/contribu­t­ors meetup #5

Thinking we could make this a mini key signing party as well [1] .. Not only would to be nice to improve our web of trust [2] - it would help future release managers too :) It would be nice to have everyone's public key uploaded to our KEYS file [3] too. Cheers -Arun [1]