Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-07-05 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-trunk-win/519/ [Jul 4, 2018 8:41:10 PM] (nanda) HDDS-212. Introduce NodeStateManager to manage the state of Datanodes in -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems

答复: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Lin,Yiqun(vip.com)
+1. -邮件原件- 发件人: Subru Krishnan [mailto:su...@apache.org] 发送时间: 2018年7月6日 5:59 收件人: Wei-Chiu Chuang 抄送: Rohith Sharma K S; Subramaniam Krishnan; yarn-dev@hadoop.apache.org; Hdfs-dev; Hadoop Common; mapreduce-...@hadoop.apache.org 主题: Re: [VOTE] reset/force push to clean up inadvertent

[jira] [Resolved] (YARN-8497) ATSv2 HBase tests failures with 'Incorrect response from timeline reader'

2018-07-05 Thread Rohith Sharma K S (JIRA)
[ https://issues.apache.org/jira/browse/YARN-8497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S resolved YARN-8497. - Resolution: Done The root cause for test failures are fixed as part YARN-8492. I am

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Eric Yang
+1 on Non-routable IP idea. My preference is to start in Hadoop-common to minimize the scope and incrementally improve. However, this will be incompatible change for initial user experience on public cloud. What would be the right release vehicle for this work (3.2+ or 4.x)? Regards, Eric

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Subru Krishnan
Unfortunately since it was a merge commit, less straightforward to revert. You can find the details in the original mail thread: http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201807.mbox/%3CCAHqguubKBqwfUMwhtJuSD7X1Bgfro_P6FV%2BhhFhMMYRaxFsF9Q%40mail.gmail.com%3E On Thu, Jul 5, 2018 at

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Wei-Chiu Chuang
I'm sorry I come to this thread late. Anu commented on INFRA-16727 saying he reverted the commit. Do we still need the vote? Thanks On Thu, Jul 5, 2018 at 2:47 PM Rohith Sharma K S wrote: > +1 > > On 5 July 2018 at 14:37, Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread varunsax...@apache.org
+1 On Fri, 6 Jul 2018 at 3:07 AM, Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Rohith Sharma K S
+1 On 5 July 2018 at 14:37, Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA sign-off

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Sunil G
+1 for this. - Sunil On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Eric Yang
+1 On 7/5/18, 2:44 PM, "Giovanni Matteo Fumarola" wrote: +1 On Thu, Jul 5, 2018 at 2:41 PM, Wangda Tan wrote: > +1 > > On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit accidentally pushed to

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Giovanni Matteo Fumarola
+1 On Thu, Jul 5, 2018 at 2:41 PM, Wangda Tan wrote: > +1 > > On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > > > Folks, > > > > There was a merge commit accidentally pushed to trunk, you can find the > > details in the mail thread [1]. > > > > I have raised an INFRA ticket [2] to

Re: [VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Wangda Tan
+1 On Thu, Jul 5, 2018 at 2:37 PM Subru Krishnan wrote: > Folks, > > There was a merge commit accidentally pushed to trunk, you can find the > details in the mail thread [1]. > > I have raised an INFRA ticket [2] to reset/force push to clean up trunk. > > Can we have a quick vote for INFRA

[VOTE] reset/force push to clean up inadvertent merge commit pushed to trunk

2018-07-05 Thread Subru Krishnan
Folks, There was a merge commit accidentally pushed to trunk, you can find the details in the mail thread [1]. I have raised an INFRA ticket [2] to reset/force push to clean up trunk. Can we have a quick vote for INFRA sign-off to proceed as this is blocking all commits? Thanks, Subru [1]

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread larry mccay
+1 from me as well. On Thu, Jul 5, 2018 at 5:19 PM, Steve Loughran wrote: > > > > On 5 Jul 2018, at 23:15, Anu Engineer wrote: > > > > +1, on the Non-Routable Idea. We like it so much that we added it to the > Ozone roadmap. > > https://issues.apache.org/jira/browse/HDDS-231 > > > > If there

[jira] [Created] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04

2018-07-05 Thread Jack Bearden (JIRA)
Jack Bearden created YARN-8498: -- Summary: Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04 Key: YARN-8498 URL: https://issues.apache.org/jira/browse/YARN-8498 Project: Hadoop YARN

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Anu Engineer
I ran “git revert -c c163d1797ade0f47d35b4a44381b8ef1dfec5b60 -m 1” that will remove all changes from Giovanni’s branch (There are 3 YARN commits). I am presuming that he can recommit the dropped changes directly into trunk. I do not know off a better way than to lose changes from his branch.

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Steve Loughran
I have the merge commit version locally; let me push it out to github OK...its on https://github.com/steveloughran/hadoop/tree/s3guard/HADOOP-15583-ddb-credentials * c163d1797ad - (HEAD -> s3guard/HADOOP-15583-ddb-credentials, gerrit/apache-ref/trunk, apache/trunk, trunk) Merge branch

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Wangda Tan
Adding back hdfs/common/mr-dev again to cc list. Here's the last merge revert commit: https://github.com/apache/hadoop/commit/39ad98903a5f042573b97a2e5438bc57af7cc7a1 On Thu, Jul 5, 2018 at 2:17 PM Wangda Tan wrote: > It looks like the latest revert is not correct, many of commits get >

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Steve Loughran
> On 5 Jul 2018, at 23:15, Anu Engineer wrote: > > +1, on the Non-Routable Idea. We like it so much that we added it to the > Ozone roadmap. > https://issues.apache.org/jira/browse/HDDS-231 > > If there is consensus on bringing this to Hadoop in general, we can build > this feature in

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Wangda Tan
It looks like the latest revert is not correct, many of commits get reverted. Dealing with merge commit revert is different from reverting a normal commit: https://www.christianengvall.se/undo-pushed-merge-git/ We have to do force reset, now it is a complete mess in trunk. On Thu, Jul 5, 2018

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Anu Engineer
+1, on the Non-Routable Idea. We like it so much that we added it to the Ozone roadmap. https://issues.apache.org/jira/browse/HDDS-231 If there is consensus on bringing this to Hadoop in general, we can build this feature in common. --Anu On 7/5/18, 1:09 PM, "Sean Busbey" wrote: I

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Anu Engineer
Based on conversations with Giovanni and Subru, I have pushed a revert for this merge. Thanks Anu On 7/5/18, 12:55 PM, "Giovanni Matteo Fumarola" wrote: + common-dev and hdfs-dev as fyi. Thanks Subru and Sean for the answer. On Thu, Jul 5, 2018 at 12:14 PM, Subru

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Vinod Kumar Vavilapalli
What is broken due to this merge commit? +Vinod > On Jul 5, 2018, at 2:03 PM, Arun Suresh wrote: > > I agree with Sean, to be honest.. it is disruptive. > Also, we have to kind of lock down the repo till it is completed.. > > I recommend we be careful and try not to get into this situation

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Wangda Tan
+ hdfs-dev/common-dev/mapreduce-dev On Thu, Jul 5, 2018 at 2:09 PM Sunil G wrote: > I just see that this is reverted. > > commit 39ad98903a5f042573b97a2e5438bc57af7cc7a1 (origin/trunk, origin/HEAD) > Author: Anu Engineer > Date: Thu Jul 5 12:22:18 2018 -0700 > > Revert "Merge branch

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Sunil G
I just see that this is reverted. commit 39ad98903a5f042573b97a2e5438bc57af7cc7a1 (origin/trunk, origin/HEAD) Author: Anu Engineer Date: Thu Jul 5 12:22:18 2018 -0700 Revert "Merge branch 'trunk' of https://git-wip-us.apache.org/repos/asf/hadoop into trunk" This reverts commit

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Arun Suresh
I agree with Sean, to be honest.. it is disruptive. Also, we have to kind of lock down the repo till it is completed.. I recommend we be careful and try not to get into this situation again.. -1 on force pushing.. Cheers -Arun On Thu, Jul 5, 2018, 1:55 PM Sean Busbey wrote: > If we need a

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Sean Busbey
If we need a vote, please have a thread with either DISCUSS or preferably VOTE in the subject so folks are more likely to see it. that said, I'm -1 (non-binding). force pushes are extremely disruptive. there's no way to know who's updated their local git repo to include these changes in the last

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Wangda Tan
+1 for force reset the branch. On Thu, Jul 5, 2018 at 12:14 PM Subru Krishnan wrote: > Looking at the merge commit, I feel it's better to reset/force push > especially since this is still the latest commit on trunk. > > I have raised an INFRA ticket requesting the same: >

Re: Why are some Jenkins builds failing in Yarn-UI

2018-07-05 Thread Steve Loughran
thanks. This is "unfortunate". I know nobody wanted this to happen, but it is still pretty serious. For the specific HADOOP-15407 branch I'd just rebase onto trunk and force-push it out, but apache git seems to be blocking me from doing that... I will keep an eye on YARN-8387; unrepeatable

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Sean Busbey
I really, really like the approach of defaulting to only non-routeable IPs allowed. it seems like a good tradeoff for complexity of implementation, pain to reconfigure, and level of protection. On Thu, Jul 5, 2018 at 2:25 PM, Todd Lipcon wrote: > The approach we took in Apache Kudu is that, if

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Giovanni Matteo Fumarola
+ common-dev and hdfs-dev as fyi. Thanks Subru and Sean for the answer. On Thu, Jul 5, 2018 at 12:14 PM, Subru Krishnan wrote: > Looking at the merge commit, I feel it's better to reset/force push > especially since this is still the latest commit on trunk. > > I have raised an INFRA ticket

[jira] [Created] (YARN-8497) ATSv2 HBase tests failures with 'Incorrect response from timeline reader'

2018-07-05 Thread Rohith Sharma K S (JIRA)
Rohith Sharma K S created YARN-8497: --- Summary: ATSv2 HBase tests failures with 'Incorrect response from timeline reader' Key: YARN-8497 URL: https://issues.apache.org/jira/browse/YARN-8497 Project:

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Todd Lipcon
The approach we took in Apache Kudu is that, if Kerberos hasn't been enabled, we default to a whitelist of subnets. The default whitelist is 127.0.0.0/8,10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,169.254.0.0/16 which matches the IANA "non-routeable IP" subnet list. In other words, out-of-the-box,

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Subru Krishnan
Looking at the merge commit, I feel it's better to reset/force push especially since this is still the latest commit on trunk. I have raised an INFRA ticket requesting the same: https://issues.apache.org/jira/browse/INFRA-16727 -S On Thu, Jul 5, 2018 at 11:45 AM, Sean Busbey wrote: > FYI, no

Re: Merge branch commit in trunk by mistake

2018-07-05 Thread Sean Busbey
FYI, no images make it through ASF mailing lists. I presume the image was of the git history? If that's correct, here's what that looks like in a paste: https://paste.apache.org/eRix There are no force pushes on trunk, so backing the change out would require the PMC asking INFRA to unblock force

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread Eric Yang
Hadoop default configuration aimed for user friendliness to increase adoption, and security can be enabled one by one. This approach is most problematic to security because system can be compromised before all security features are turned on. Larry's proposal will add some safety to remind

Merge branch commit in trunk by mistake

2018-07-05 Thread Giovanni Matteo Fumarola
Hi folks, After I pushed something on trunk a merge commit showed up in the history. *My bad*. Since it was one of my first patches, I run a few tests on my machine before checked in. While I was running all the tests, someone else checked in. I correctly pulled all the new changes. Even

Re: Why are some Jenkins builds failing in Yarn-UI

2018-07-05 Thread Sunil G
Hi Steve I was recently looking into this failure and this is due to a repo hosting problem. It seems Bower team has deprecated old registry url https://bower.herokuapp.com and need to change registry url to https://registry.bower.io. YARN-8457 addressed this problem is backported till

Re: [DISCUSS]: securing ASF Hadoop releases out of the box

2018-07-05 Thread larry mccay
Hi Steve - This is a long overdue DISCUSS thread! Perhaps the UIs can very visibly state (in red) "WARNING: UNSECURED UI ACCESS - OPEN TO COMPROMISE" - maybe even force a click through the warning to get to the page like SSL exceptions in the browser do? Similar tactic for UI access without SSL?

Why are some Jenkins builds failing in Yarn-UI

2018-07-05 Thread Steve Loughran
Hi The HADOOP-15407 "abfs" branch has started failing https://builds.apache.org/job/PreCommit-HADOOP-Build/14861/artifact/out/patch-compile-root.txt [INFO] --- frontend-maven-plugin:1.5:bower (bower install) @ hadoop-yarn-ui --- [INFO] Running 'bower install' in

[jira] [Created] (YARN-8496) The capacity scheduler uses label to cause vcore to be incorrect

2018-07-05 Thread tangshangwen (JIRA)
tangshangwen created YARN-8496: -- Summary: The capacity scheduler uses label to cause vcore to be incorrect Key: YARN-8496 URL: https://issues.apache.org/jira/browse/YARN-8496 Project: Hadoop YARN

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-07-05 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-trunk-win/518/ [Jul 3, 2018 8:50:11 PM] (fabbri) HADOOP-15215 s3guard set-capacity command to fail on read/write of 0 [Jul 3, 2018 9:11:52 PM] (aengineer) HDDS-175. Refactor ContainerInfo to remove Pipeline object from it. [Jul 4, 2018

[jira] [Created] (YARN-8495) Priority scheduling support in FairShare scheduler

2018-07-05 Thread Dillon Zhang (JIRA)
Dillon Zhang created YARN-8495: -- Summary: Priority scheduling support in FairShare scheduler Key: YARN-8495 URL: https://issues.apache.org/jira/browse/YARN-8495 Project: Hadoop YARN Issue Type:

[jira] [Created] (YARN-8494) TestQueueManagementDynamicEditPolicy.testEditSchedule is flaky

2018-07-05 Thread Wilfred Spiegelenburg (JIRA)
Wilfred Spiegelenburg created YARN-8494: --- Summary: TestQueueManagementDynamicEditPolicy.testEditSchedule is flaky Key: YARN-8494 URL: https://issues.apache.org/jira/browse/YARN-8494 Project: