Re: [VOTE] Merge yarn-native-services branch into trunk

> If it doesn’t have all the bells and whistles, then it shouldn’t be on > port 53 by default. Sure, I’ll change the default port to not use 53 and document it. > *how* is it getting launched on a privileged port? It sounds like the > expectation is to run “command” as root. *ALL*

Re: [DISCUSS] Looking to a 2.9.0 release

Hi Subru, Thanks for starting the discussion. We are targeting merging YARN-5734 (API-based scheduler configuration) to branch-2 before the release of 2.9.0, since the feature is close to complete. Regarding the requirements for merge, 1. API compatibility - this feature adds new APIs, does not

Re: [VOTE] Merge yarn-native-services branch into trunk

Thanks Allen. You are right, the github renderer does have trouble rendering the headers. I was only looking at the html generated by mvn site, which did not have trouble rendering them. Anyway I added a space after all the hashes and it looks ok through github now. -Gour On 9/5/17, 3:20 PM,

Re: [VOTE] Merge yarn-native-services branch into trunk

> On Sep 5, 2017, at 3:12 PM, Gour Saha wrote: > > 2) Lots of markdown problems in the NativeServicesDiscovery.md document. > This includes things like Œyarnsite.xml¹ (missing a dash.) > > The md patch uploaded to YARN-5244 had some special chars. I fixed those > in

Re: [VOTE] Merge yarn-native-services branch into trunk

2) Lots of markdown problems in the NativeServicesDiscovery.md document. This includes things like Œyarnsite.xml¹ (missing a dash.) The md patch uploaded to YARN-5244 had some special chars. I fixed those in YARN-7161. > -

Re: [VOTE] Merge yarn-native-services branch into trunk

> On Sep 5, 2017, at 2:53 PM, Jian He wrote: > >> Based on the documentation, this doesn’t appear to be a fully function DNS >> server as an admin would expect (e.g., BIND, Knot, whatever). Where’s >> forwarding? How do I setup notify? Are secondaries even supported?

Re: Apache Hadoop 2.8.2 Release Plan

I assume the quiet over the holiday means we agreed to move forward without taking HADOOP-14439 into 2.8.2. There is a new release building (docker based) issue could be related to HADOOP-14474 where we removed oracle java 7 installer due to recent download address/contract change by Oracle.

Re: [VOTE] Merge yarn-native-services branch into trunk

> 1) Did I miss it or is there no actual end-user documentation on how to use > this? Yes, we are in the process of finishing up the the doc and posting it. We considered this a release blocker for 3.0.0-beta1 and so working on it in parallel while the branch merge happens. > 2) Lots

[jira] [Created] (HADOOP-14842) Hadoop 2.8.2 release build process get stuck due to java issue

Junping Du created HADOOP-14842: --- Summary: Hadoop 2.8.2 release build process get stuck due to java issue Key: HADOOP-14842 URL: https://issues.apache.org/jira/browse/HADOOP-14842 Project: Hadoop

[jira] [Resolved] (HADOOP-13998) Merge initial S3guard release into trunk

[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HADOOP-13998. -- Resolution: Done Re-resolving per above. > Merge initial S3guard release into trunk >

[jira] [Reopened] (HADOOP-13998) Merge initial S3guard release into trunk

[ https://issues.apache.org/jira/browse/HADOOP-13998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang reopened HADOOP-13998: -- Re-opening to resolve as "Complete" or something, since this code change was attributed to the

[jira] [Created] (HADOOP-14841) Add KMS Client retry to handle 'No content to map' EOFExceptions

Xiao Chen created HADOOP-14841: -- Summary: Add KMS Client retry to handle 'No content to map' EOFExceptions Key: HADOOP-14841 URL: https://issues.apache.org/jira/browse/HADOOP-14841 Project: Hadoop

Re: [VOTE] Merge yarn-native-services branch into trunk

> On Aug 31, 2017, at 8:33 PM, Jian He wrote: > I would like to call a vote for merging yarn-native-services to trunk. 1) Did I miss it or is there no actual end-user documentation on how to use this? I see

Re: [DISCUSS] Merge yarn-native-services branch into trunk

Thanks for all the work on this folks. I know the VOTE thread has started for this. But I did have a couple of questions: * Is there a branch-2 merge planned for this ? * I understand YARN-7126 has some introductory documentation, But I think we need to flesh it up a bit more before release, I

[jira] [Created] (HADOOP-14840) Tool to estimate resource requirements of an application pipeline based on prior executions

Subru Krishnan created HADOOP-14840: --- Summary: Tool to estimate resource requirements of an application pipeline based on prior executions Key: HADOOP-14840 URL:

Re: [VOTE] Merge yarn-native-services branch into trunk

+1 (binding). Cheers -Arun On Fri, Sep 1, 2017 at 5:24 PM, Wangda Tan wrote: > +1 (Binding), I tried to use YARN service assembly before to run different > kinds of jobs (for example, distributed Tensorflow), it is really easy for > end user to run jobs on YARN. > > Thanks

Re: DISCUSS: Hadoop Compatability Guidelines

Thanks for starting this Daniel. I think we should also add a section for store compatibility (all state stores including RM, NM, Federation etc.). Essentially an explicit policy detailing when is it ok to change the major and minor versions and how it should relate to the hadoop release version.

Re: DISCUSS: Hadoop Compatability Guidelines

Good idea.  I should have thought of that. :)  Done. Daniel On 9/5/17 10:33 AM, Anu Engineer wrote: Could you please attach the PDFs to the JIRA. I think the mailer is stripping them off from the mail. Thanks Anu On 9/5/17, 9:44 AM, "Daniel Templeton" wrote:

Re: DISCUSS: Hadoop Compatability Guidelines

Could you please attach the PDFs to the JIRA. I think the mailer is stripping them off from the mail. Thanks Anu On 9/5/17, 9:44 AM, "Daniel Templeton" wrote: >Resending with a broader audience, and reattaching the PDFs. > >Daniel > >On 9/4/17 9:01 AM, Daniel

[jira] [Resolved] (HADOOP-14828) RetryUpToMaximumTimeWithFixedSleep is not bounded by maximum time

[ https://issues.apache.org/jira/browse/HADOOP-14828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung resolved HADOOP-14828. Resolution: Duplicate > RetryUpToMaximumTimeWithFixedSleep is not bounded by maximum time

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/514/ [Sep 4, 2017 2:41:02 PM] (varunsaxena) YARN-7152. [ATSv2] Registering timeline client before AMRMClient service [Sep 5, 2017 2:36:43 AM] (sunilg) YARN-7022. Improve click interaction in queue topology in

Re: DISCUSS: Hadoop Compatability Guidelines

Resending with a broader audience, and reattaching the PDFs. Daniel On 9/4/17 9:01 AM, Daniel Templeton wrote: All, in prep for Hadoop 3 beta 1 I've been working on updating the compatibility guidelines on HADOOP-13714.  I think the initial doc is more or less complete, so I'd like to open

[jira] [Created] (HADOOP-14837) Handle S3A "glacier" data

Steve Loughran created HADOOP-14837: --- Summary: Handle S3A "glacier" data Key: HADOOP-14837 URL: https://issues.apache.org/jira/browse/HADOOP-14837 Project: Hadoop Common Issue Type:

[jira] [Created] (HADOOP-14836) multiple versions of maven-clean-plugin in use

Allen Wittenauer created HADOOP-14836: - Summary: multiple versions of maven-clean-plugin in use Key: HADOOP-14836 URL: https://issues.apache.org/jira/browse/HADOOP-14836 Project: Hadoop Common

[jira] [Created] (HADOOP-14835) mvn site build throws SAX errors

Allen Wittenauer created HADOOP-14835: - Summary: mvn site build throws SAX errors Key: HADOOP-14835 URL: https://issues.apache.org/jira/browse/HADOOP-14835 Project: Hadoop Common Issue

[jira] [Reopened] (HADOOP-14736) S3AInputStream to implement an efficient skip() call through seeking

[ https://issues.apache.org/jira/browse/HADOOP-14736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HADOOP-14736: - > S3AInputStream to implement an efficient skip() call through seeking >

[jira] [Created] (HADOOP-14834) Make default output stream of S3a the block output stream

Steve Loughran created HADOOP-14834: --- Summary: Make default output stream of S3a the block output stream Key: HADOOP-14834 URL: https://issues.apache.org/jira/browse/HADOOP-14834 Project: Hadoop

[jira] [Resolved] (HADOOP-14826) review S3 docs prior to 3.0-beta-1

[ https://issues.apache.org/jira/browse/HADOOP-14826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14826. - Resolution: Duplicate > review S3 docs prior to 3.0-beta-1 >

[jira] [Resolved] (HADOOP-13630) split up AWS index.md

[ https://issues.apache.org/jira/browse/HADOOP-13630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13630. - Resolution: Duplicate > split up AWS index.md > - > >

[jira] [Resolved] (HADOOP-14736) S3AInputStream to implement an efficient skip() call through seeking

[ https://issues.apache.org/jira/browse/HADOOP-14736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14736. - Resolution: Duplicate > S3AInputStream to implement an efficient skip() call through

[jira] [Resolved] (HADOOP-14605) transient list consistency failure in ITestS3AContractRootDir

[ https://issues.apache.org/jira/browse/HADOOP-14605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-14605. - Resolution: Duplicate closing as a duplicate of HADOOP-13271. It's a slightly different

[jira] [Resolved] (HADOOP-13950) S3A create(path, overwrite=true) need only check for path being a dir, not a file

[ https://issues.apache.org/jira/browse/HADOOP-13950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13950. - Resolution: Duplicate > S3A create(path, overwrite=true) need only check for path being

[jira] [Resolved] (HADOOP-13757) Remove verifyBuckets overhead in S3AFileSystem::initialize()

[ https://issues.apache.org/jira/browse/HADOOP-13757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-13757. - Resolution: Won't Fix I'm going to make this a wontfix, with s3guard & c logging in has

[jira] [Created] (HADOOP-14833) Move s3a user:secret auth out of the main chain of auth mechanisms

Steve Loughran created HADOOP-14833: --- Summary: Move s3a user:secret auth out of the main chain of auth mechanisms Key: HADOOP-14833 URL: https://issues.apache.org/jira/browse/HADOOP-14833 Project: