Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-07-05 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-trunk-win/519/ [Jul 4, 2018 8:41:10 PM] (nanda) HDDS-212. Introduce NodeStateManager to manage the state of Datanodes in -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems

答复: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Lin,Yiqun(vip.com)
+1. -邮件原件- 发件人: Subru Krishnan [mailto:su...@apache.org] 发送时间: 2018年7月6日 5:59 收件人: Wei-Chiu Chuang 抄送: Rohith Sharma K S; Subramaniam Krishnan; yarn-...@hadoop.apache.org; Hdfs-dev; Hadoop Common; mapreduce-...@hadoop.apache.org 主题: Re: [VOTE] reset/force push to clean up inadvertent

[jira] [Created] (HADOOP-15585) Fix spaces in HADOOP_OPTS arguments

2018-07-05 Thread Brandon Scheller (JIRA)
Brandon Scheller created HADOOP-15585: - Summary: Fix spaces in HADOOP_OPTS arguments Key: HADOOP-15585 URL: https://issues.apache.org/jira/browse/HADOOP-15585 Project: Hadoop Common

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Eric Yang
+1 on Non-routable IP idea. My preference is to start in Hadoop-common to minimize the scope and incrementally improve. However, this will be incompatible change for initial user experience on public cloud. What would be the right release vehicle for this work (3.2+ or 4.x)? Regards, Eric

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Subru Krishnan
Unfortunately since it was a merge commit, less straightforward to revert. You can find the details in the original mail thread: http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201807.mbox/%3CCAHqguubKBqwfUMwhtJuSD7X1Bgfro_P6FV%2BhhFhMMYRaxFsF9Q%40mail.gmail.com%3E On Thu, Jul 5, 2018 at

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Wei-Chiu Chuang
I'm sorry I come to this thread late. Anu commented on INFRA-16727 saying he reverted the commit. Do we still need the vote? Thanks On Thu, Jul 5, 2018 at 2:47 PM Rohith Sharma K S wrote: > +1 > > On 5 July 2018 at 14:37, Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread varunsax...@apache.org
+1 On Fri, 6 Jul 2018 at 3:07 AM, Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Rohith Sharma K S
+1 On 5 July 2018 at 14:37, Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA sign-off

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Sunil G
+1 for this. - Sunil On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Eric Yang
+1 On 7/5/18, 2:44 PM, "Giovanni Matteo Fumarola" wrote: +1 On Thu, Jul 5, 2018 at 2:41 PM, Wangda Tan wrote: > +1 > > On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit accidentally pushed to

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Giovanni Matteo Fumarola
+1 On Thu, Jul 5, 2018 at 2:41 PM, Wangda Tan wrote: > +1 > > On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit accidentally pushed to trunk, you can find the > > details in the mail thread [1]. > > > > I have raised an INFRA ticket [2] to

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Wangda Tan
+1 On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA

[VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Subru Krishnan
Folks, There was a merge commit accidentally pushed to trunk, you can find the details in the mail thread [1]. I have raised an INFRA ticket [2] to reset/force push to clean up trunk. Can we have a quick vote for INFRA sign-off to proceed as this is blocking all commits? Thanks, Subru [1]

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread larry mccay
+1 from me as well. On Thu, Jul 5, 2018 at 5:19 PM, Steve Loughran wrote: > > > > On 5 Jul 2018, at 23:15, Anu Engineer wrote: > > > > +1, on the Non-Routable Idea. We like it so much that we added it to the > Ozone roadmap. > > https://issues.apache.org/jira/browse/HDDS-231 > > > > If there

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Anu Engineer
I ran “git revert -c c163d1797ade0f47d35b4a44381b8ef1dfec5b60 -m 1” that will remove all changes from Giovanni’s branch (There are 3 YARN commits). I am presuming that he can recommit the dropped changes directly into trunk. I do not know off a better way than to lose changes from his branch.

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Steve Loughran
> On 5 Jul 2018, at 23:15, Anu Engineer wrote: > > +1, on the Non-Routable Idea. We like it so much that we added it to the > Ozone roadmap. > https://issues.apache.org/jira/browse/HDDS-231 > > If there is consensus on bringing this to Hadoop in general, we can build > this feature in

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Wangda Tan
Adding back hdfs/common/mr-dev again to cc list. Here's the last merge revert commit: https://github.com/apache/hadoop/commit/39ad98903a5f042573b97a2e5438bc57af7cc7a1 On Thu, Jul 5, 2018 at 2:17 PM Wangda Tan wrote: > It looks like the latest revert is not correct, many of commits get >

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Anu Engineer
+1, on the Non-Routable Idea. We like it so much that we added it to the Ozone roadmap. https://issues.apache.org/jira/browse/HDDS-231 If there is consensus on bringing this to Hadoop in general, we can build this feature in common. --Anu On 7/5/18, 1:09 PM, "Sean Busbey" wrote: I

[jira] [Created] (HADOOP-15584) move httpcomponents version in pom.xml

2018-07-05 Thread Brandon Scheller (JIRA)
Brandon Scheller created HADOOP-15584: - Summary: move httpcomponents version in pom.xml Key: HADOOP-15584 URL: https://issues.apache.org/jira/browse/HADOOP-15584 Project: Hadoop Common

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Wangda Tan
+ hdfs-dev/common-dev/mapreduce-dev On Thu, Jul 5, 2018 at 2:09 PM Sunil G wrote: > I just see that this is reverted. > > commit 39ad98903a5f042573b97a2e5438bc57af7cc7a1 (origin/trunk, origin/HEAD) > Author: Anu Engineer > Date: Thu Jul 5 12:22:18 2018 -0700 > > Revert "Merge branch

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Anu Engineer
Based on conversations with Giovanni and Subru, I have pushed a revert for this merge. Thanks Anu On 7/5/18, 12:55 PM, "Giovanni Matteo Fumarola" wrote: + common-dev and hdfs-dev as fyi. Thanks Subru and Sean for the answer. On Thu, Jul 5, 2018 at 12:14 PM, Subru

[jira] [Created] (HADOOP-15583) S3Guard to get AWS Credential chain from S3AFS

2018-07-05 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15583: --- Summary: S3Guard to get AWS Credential chain from S3AFS Key: HADOOP-15583 URL: https://issues.apache.org/jira/browse/HADOOP-15583 Project: Hadoop Common

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Sean Busbey
I really, really like the approach of defaulting to only non-routeable IPs allowed. it seems like a good tradeoff for complexity of implementation, pain to reconfigure, and level of protection. On Thu, Jul 5, 2018 at 2:25 PM, Todd Lipcon wrote: > The approach we took in Apache Kudu is that, if

Re: Why are some Jenkins builds failing in Yarn-UI

2018-07-05 Thread Steve Loughran
thanks. This is "unfortunate". I know nobody wanted this to happen, but it is still pretty serious. For the specific HADOOP-15407 branch I'd just rebase onto trunk and force-push it out, but apache git seems to be blocking me from doing that... I will keep an eye on YARN-8387; unrepeatable

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Giovanni Matteo Fumarola
+ common-dev and hdfs-dev as fyi. Thanks Subru and Sean for the answer. On Thu, Jul 5, 2018 at 12:14 PM, Subru Krishnan wrote: > Looking at the merge commit, I feel it's better to reset/force push > especially since this is still the latest commit on trunk. > > I have raised an INFRA ticket

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Todd Lipcon
The approach we took in Apache Kudu is that, if Kerberos hasn't been enabled, we default to a whitelist of subnets. The default whitelist is 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,169.254.0.0/16 which matches the IANA "non-routeable IP" subnet list. In other words, out-of-the-box,

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Eric Yang
Hadoop default configuration aimed for user friendliness to increase adoption, and security can be enabled one by one. This approach is most problematic to security because system can be compromised before all security features are turned on. Larry's proposal will add some safety to remind

Re: Why are some Jenkins builds failing in Yarn-UI

2018-07-05 Thread Sunil G
Hi Steve I was recently looking into this failure and this is due to a repo hosting problem. It seems Bower team has deprecated old registry url https://bower.herokuapp.com and need to change registry url to https://registry.bower.io. YARN-8457 addressed this problem is backported till

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread larry mccay
Hi Steve - This is a long overdue DISCUSS thread! Perhaps the UIs can very visibly state (in red) "WARNING: UNSECURED UI ACCESS - OPEN TO COMPROMISE" - maybe even force a click through the warning to get to the page like SSL exceptions in the browser do? Similar tactic for UI access without SSL?

Why are some Jenkins builds failing in Yarn-UI

2018-07-05 Thread Steve Loughran
Hi The HADOOP-15407 "abfs" branch has started failing https://builds.apache.org/job/PreCommit-HADOOP-Build/14861/artifact/out/patch-compile-root.txt [INFO] --- frontend-maven-plugin:1.5:bower (bower install) @ hadoop-yarn-ui --- [INFO] Running 'bower install' in

[jira] [Created] (HADOOP-15582) Document ABFS

2018-07-05 Thread Steve Loughran (JIRA)
Steve Loughran created HADOOP-15582: --- Summary: Document ABFS Key: HADOOP-15582 URL: https://issues.apache.org/jira/browse/HADOOP-15582 Project: Hadoop Common Issue Type: Sub-task

[jira] [Resolved] (HADOOP-15545) ABFS initialize() throws string out of bounds exception of the URI isn't fully qualified

2018-07-05 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-15545?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-15545. - Resolution: Duplicate Assignee: Steve Loughran Fix Version/s: 3.2.0

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-07-05 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-trunk-win/518/ [Jul 3, 2018 8:50:11 PM] (fabbri) HADOOP-15215 s3guard set-capacity command to fail on read/write of 0 [Jul 3, 2018 9:11:52 PM] (aengineer) HDDS-175. Refactor ContainerInfo to remove Pipeline object from it. [Jul 4, 2018