date:20160707

[jira] [Commented] (HDFS-10336) TestBalancer failing intermittently because of not reseting UserGroupInformation completely

2016-07-07 Thread Xiao Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367222#comment-15367222
 ] 

Xiao Chen commented on HDFS-10336:
--

+1 (non-binding) on patch 3. Thanks, [~linyiqun].

> TestBalancer failing intermittently because of not reseting 
> UserGroupInformation completely
> ---
>
> Key: HDFS-10336
> URL: https://issues.apache.org/jira/browse/HDFS-10336
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.0.0-alpha1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HDFS-10336.001.patch, HDFS-10336.002.patch, 
> HDFS-10336.003.patch
>
>
> The unit test {{TestBalancer}} failed sometimes. 
> I looked for the reason. I found two main reasons causing this.
> * 1st. The test {{TestBalancer#testBalancerWithKeytabs}} executed timeout.
> {code}
> org.apache.hadoop.hdfs.server.balancer.TestBalancer
> testBalancerWithKeytabs(org.apache.hadoop.hdfs.server.balancer.TestBalancer)  
> Time elapsed: 300.41 sec  <<< ERROR!
> java.lang.Exception: test timed out after 30 milliseconds
>   at java.lang.Thread.sleep(Native Method)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.waitForMoveCompletion(Dispatcher.java:1122)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchBlockMoves(Dispatcher.java:1096)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchAndCheckContinue(Dispatcher.java:1060)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.runOneIteration(Balancer.java:635)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:689)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.testUnknownDatanode(TestBalancer.java:1098)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.access$000(TestBalancer.java:125)
> {code}
> * 2nd. The test {{TestBalancer#testBalancerWithKeytabs}} reset the {{UGI}} 
> not completely sometimes in the finally block. And this affected the other 
> unit tests threw {{IOException}}, like this:
> {code}
> testBalancerWithNonZeroThreadsForMove(org.apache.hadoop.hdfs.server.balancer.TestBalancer)
>   Time elapsed: 0 sec  <<< ERROR!
> java.io.IOException: Running in secure mode, but config doesn't have a keytab
>   at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:300)
> {code}
> And there were not only one test will be affected by this. We should add a 
> line to do before doing reset {{UGI}} operation and can avoid the potenial 
> exception happens.
> {code}
> UserGroupInformation.reset();
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Anu Engineer (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-10579:

Attachment: HDFS-10579.003.patch

This patch is depends on HADOOP-13352. Posting here for early code review. Once 
HADOOP-13352 is committed, this patch can be submitted for jenkins.

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch, 
> HDFS-10579.003.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Anu Engineer (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-10579:

Status: Open  (was: Patch Available)

Cancelling patch since this patch depends on HADOOP-13352

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10336) TestBalancer failing intermittently because of not reseting UserGroupInformation completely

2016-07-07 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367195#comment-15367195
 ] 

Hadoop QA commented on HDFS-10336:
--

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
27s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
40s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 73m 
12s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 95m  2s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816744/HDFS-10336.003.patch |
| JIRA Issue | HDFS-10336 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux e3ea115e5ee6 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 9d46a49 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16004/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16004/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> TestBalancer failing intermittently because of not reseting 
> UserGroupInformation completely
> ---
>
> Key: HDFS-10336
> URL: https://issues.apache.org/jira/browse/HDFS-10336
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.0.0-alpha1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HDFS-10336.001.patch, HDFS-10336.002.patch, 
> HDFS-10336.003.patch
>
>
> The unit test {{TestBalancer}} failed sometimes. 
>

[jira] [Commented] (HDFS-10336) TestBalancer failing intermittently because of not reseting UserGroupInformation completely

2016-07-07 Thread Yiqun Lin (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367104#comment-15367104
 ] 

Yiqun Lin commented on HDFS-10336:
--

Hi, [~xiaochen], as you can see that the unit test 
{{TestBalancer#testBalancerWithKeytabs}}  will be timeout in 300s  which was 
memtioned in description. I think 10 mins is a long enough time, because the 
test only costs around 15 seconds in my local. If it still happens timeout in 
these tests, maybe we would like to do further optimization ranther than 
increasing the timeout. I suggest that we can do the optimization work in a 
separate jira. This jira is focus on the reseting UGI in the test that we can 
see from the title. Post a new patch for addressing the comments.

> TestBalancer failing intermittently because of not reseting 
> UserGroupInformation completely
> ---
>
> Key: HDFS-10336
> URL: https://issues.apache.org/jira/browse/HDFS-10336
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.0.0-alpha1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HDFS-10336.001.patch, HDFS-10336.002.patch, 
> HDFS-10336.003.patch
>
>
> The unit test {{TestBalancer}} failed sometimes. 
> I looked for the reason. I found two main reasons causing this.
> * 1st. The test {{TestBalancer#testBalancerWithKeytabs}} executed timeout.
> {code}
> org.apache.hadoop.hdfs.server.balancer.TestBalancer
> testBalancerWithKeytabs(org.apache.hadoop.hdfs.server.balancer.TestBalancer)  
> Time elapsed: 300.41 sec  <<< ERROR!
> java.lang.Exception: test timed out after 30 milliseconds
>   at java.lang.Thread.sleep(Native Method)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.waitForMoveCompletion(Dispatcher.java:1122)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchBlockMoves(Dispatcher.java:1096)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchAndCheckContinue(Dispatcher.java:1060)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.runOneIteration(Balancer.java:635)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:689)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.testUnknownDatanode(TestBalancer.java:1098)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.access$000(TestBalancer.java:125)
> {code}
> * 2nd. The test {{TestBalancer#testBalancerWithKeytabs}} reset the {{UGI}} 
> not completely sometimes in the finally block. And this affected the other 
> unit tests threw {{IOException}}, like this:
> {code}
> testBalancerWithNonZeroThreadsForMove(org.apache.hadoop.hdfs.server.balancer.TestBalancer)
>   Time elapsed: 0 sec  <<< ERROR!
> java.io.IOException: Running in secure mode, but config doesn't have a keytab
>   at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:300)
> {code}
> And there were not only one test will be affected by this. We should add a 
> line to do before doing reset {{UGI}} operation and can avoid the potenial 
> exception happens.
> {code}
> UserGroupInformation.reset();
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10336) TestBalancer failing intermittently because of not reseting UserGroupInformation completely

2016-07-07 Thread Yiqun Lin (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yiqun Lin updated HDFS-10336:
-
Attachment: HDFS-10336.003.patch

> TestBalancer failing intermittently because of not reseting 
> UserGroupInformation completely
> ---
>
> Key: HDFS-10336
> URL: https://issues.apache.org/jira/browse/HDFS-10336
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.0.0-alpha1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HDFS-10336.001.patch, HDFS-10336.002.patch, 
> HDFS-10336.003.patch
>
>
> The unit test {{TestBalancer}} failed sometimes. 
> I looked for the reason. I found two main reasons causing this.
> * 1st. The test {{TestBalancer#testBalancerWithKeytabs}} executed timeout.
> {code}
> org.apache.hadoop.hdfs.server.balancer.TestBalancer
> testBalancerWithKeytabs(org.apache.hadoop.hdfs.server.balancer.TestBalancer)  
> Time elapsed: 300.41 sec  <<< ERROR!
> java.lang.Exception: test timed out after 30 milliseconds
>   at java.lang.Thread.sleep(Native Method)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.waitForMoveCompletion(Dispatcher.java:1122)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchBlockMoves(Dispatcher.java:1096)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchAndCheckContinue(Dispatcher.java:1060)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.runOneIteration(Balancer.java:635)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:689)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.testUnknownDatanode(TestBalancer.java:1098)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.access$000(TestBalancer.java:125)
> {code}
> * 2nd. The test {{TestBalancer#testBalancerWithKeytabs}} reset the {{UGI}} 
> not completely sometimes in the finally block. And this affected the other 
> unit tests threw {{IOException}}, like this:
> {code}
> testBalancerWithNonZeroThreadsForMove(org.apache.hadoop.hdfs.server.balancer.TestBalancer)
>   Time elapsed: 0 sec  <<< ERROR!
> java.io.IOException: Running in secure mode, but config doesn't have a keytab
>   at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:300)
> {code}
> And there were not only one test will be affected by this. We should add a 
> line to do before doing reset {{UGI}} operation and can avoid the potenial 
> exception happens.
> {code}
> UserGroupInformation.reset();
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Anu Engineer (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367032#comment-15367032
 ] 

Anu Engineer commented on HDFS-10579:
-

[~haibochen] HDFS JIRAs cannot be made sub-tasks of COMMON jiras. I have made 
this JIRA dependent on the JIRA in common.


> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10534) NameNode WebUI should display DataNode usage histogram

2016-07-07 Thread Kai Sasaki (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367018#comment-15367018
 ] 

Kai Sasaki commented on HDFS-10534:
---

{{d3.min.js}} is required to render histogram. Since the file is minified, the 
name is {{d3.min}}. 
But if we can use non-minified version of d3.js, it can be formatted and will 
be applied. I'll try to replace it.

> NameNode WebUI should display DataNode usage histogram
> --
>
> Key: HDFS-10534
> URL: https://issues.apache.org/jira/browse/HDFS-10534
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: namenode, ui
>Reporter: Zhe Zhang
>Assignee: Kai Sasaki
> Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, 
> HDFS-10534.03.patch, HDFS-10534.04.patch, HDFS-10534.05.patch, 
> HDFS-10534.06.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png, Screen Shot 
> 2016-07-07 at 23.29.14.png
>
>
> In addition of *Min/Median/Max*, another meaningful metric for cluster 
> balance is DN usage in histogram style.
> Since NN already has provided necessary information to calculate histogram of 
> DN usage, it can be done in JS side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-07-07 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367015#comment-15367015
 ] 

Hadoop QA commented on HDFS-5805:
-

| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
38s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 62m 
53s{color} | {color:green} hadoop-hdfs in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 82m 23s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816718/HDFS-5805.001.patch |
| JIRA Issue | HDFS-5805 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 1e580629db01 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / 9d46a49 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16003/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16003/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> TestCheckpoint.testCheckpoint fails intermittently on branch2
> -
>
> Key: HDFS-5805
> URL: https://issues.apache.org/jira/browse/HDFS-5805
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Mit Desai
>Assignee: Eric Badger
> Attachments: HDFS-5805.001.patch
>
>
> {noformat}
> java.lang.AssertionError: Bad value for metric GetEditAvgTime
> Expected: gt(0.0)
>  got: <0.0>
>   at org.junit.Assert.assertThat(Assert.java:780)
>   at 
>

[jira] [Commented] (HDFS-10512) VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks

2016-07-07 Thread Yongjun Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15367012#comment-15367012
 ] 

Yongjun Zhang commented on HDFS-10512:
--

Thanks [~jojochuang] and [~linyiqun] for working on the issue and [~ajisakaa] 
for the review.

I looked at the patch, and have one question:

The patch changed from 
{code}
datanode.reportBadBlocks(new ExtendedBlock(bpid, corruptBlock));
{code}
to
{code}
datanode.reportBadBlocks(new ExtendedBlock(bpid, memBlockInfo),
memBlockInfo.getVolume());
{code}

where the second parameter of constructor {{ExtendedBlock}} was changed from 
{{corruptBlock}} to {{memBlockInfo}}. As we know, the block size recorded in 
{{corruptBlock}} and {{memBlockInfo}} are different per the following code:

{code}
  // Compare block size
  if (memBlockInfo.getNumBytes() != memFile.length()) {
// Update the length based on the block file
corruptBlock = new Block(memBlockInfo);
LOG.warn("Updating size of block " + blockId + " from "
+ memBlockInfo.getNumBytes() + " to " + memFile.length());
memBlockInfo.setNumBytes(memFile.length());
  }
{code}

When reporting the bad block, do we intend to report the new length or the old 
length back to NN (the old code reported the old length, the patch reported the 
new length)?

I can see this might not be a real issue, just want to point it out, Is it 
intended change? I guess passing either  {{corruptBlock}} or {{memBlockInfo}} 
to  the second parameter of constructor {{ExtendedBlock}}  is fine.

Would you guys please comment?

Other than that, the patch looks good to me.

Thanks.



> VolumeScanner may terminate due to NPE in DataNode.reportBadBlocks
> --
>
> Key: HDFS-10512
> URL: https://issues.apache.org/jira/browse/HDFS-10512
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode
>Reporter: Wei-Chiu Chuang
>Assignee: Yiqun Lin
> Attachments: HDFS-10512.001.patch, HDFS-10512.002.patch, 
> HDFS-10512.004.patch, HDFS-10512.005.patch
>
>
> VolumeScanner may terminate due to unexpected NullPointerException thrown in 
> {{DataNode.reportBadBlocks()}}. This is different from HDFS-8850/HDFS-9190
> I observed this bug in a production CDH 5.5.1 cluster and the same bug still 
> persist in upstream trunk.
> {noformat}
> 2016-04-07 20:30:53,830 WARN 
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner: Reporting bad 
> BP-1800173197-10.204.68.5-125156296:blk_1170134484_96468685 on /dfs/dn
> 2016-04-07 20:30:53,831 ERROR 
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner: VolumeScanner(/dfs/dn, 
> DS-89b72832-2a8c-48f3-8235-48e6c5eb5ab3) exiting because of exception
> java.lang.NullPointerException
> at 
> org.apache.hadoop.hdfs.server.datanode.DataNode.reportBadBlocks(DataNode.java:1018)
> at 
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner$ScanResultHandler.handle(VolumeScanner.java:287)
> at 
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner.scanBlock(VolumeScanner.java:443)
> at 
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner.runLoop(VolumeScanner.java:547)
> at 
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner.run(VolumeScanner.java:621)
> 2016-04-07 20:30:53,832 INFO 
> org.apache.hadoop.hdfs.server.datanode.VolumeScanner: VolumeScanner(/dfs/dn, 
> DS-89b72832-2a8c-48f3-8235-48e6c5eb5ab3) exiting.
> {noformat}
> I think the NPE comes from the volume variable in the following code snippet. 
> Somehow the volume scanner know the volume, but the datanode can not lookup 
> the volume using the block.
> {code}
> public void reportBadBlocks(ExtendedBlock block) throws IOException{
> BPOfferService bpos = getBPOSForBlock(block);
> FsVolumeSpi volume = getFSDataset().getVolume(block);
> bpos.reportBadBlocks(
> block, volume.getStorageID(), volume.getStorageType());
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-9310) TestDataNodeHotSwapVolumes fails occasionally

2016-07-07 Thread Lei (Eddy) Xu (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-9310?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei (Eddy) Xu resolved HDFS-9310.
-
   Resolution: Duplicate
 Assignee: Lei (Eddy) Xu
Fix Version/s: 2.8.0

It was fixed in HDFS-9137.

> TestDataNodeHotSwapVolumes fails occasionally
> -
>
> Key: HDFS-9310
> URL: https://issues.apache.org/jira/browse/HDFS-9310
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 2.8.0
>Reporter: Arpit Agarwal
>Assignee: Lei (Eddy) Xu
> Fix For: 2.8.0
>
>
> TestDataNodeHotSwapVolumes fails occasionally in Jenkins and locally. e.g. 
> https://builds.apache.org/job/PreCommit-HDFS-Build/13197/testReport/org.apache.hadoop.hdfs.server.datanode/TestDataNodeHotSwapVolumes/testRemoveVolumeBeingWritten/
> *Error Message*
> Timed out waiting for /test to reach 3 replicas
> *Stacktrace*
> java.util.concurrent.TimeoutException: Timed out waiting for /test to reach 3 
> replicas
>   at 
> org.apache.hadoop.hdfs.DFSTestUtil.waitReplication(DFSTestUtil.java:768)
>   at 
> org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWrittenForDatanode(TestDataNodeHotSwapVolumes.java:644)
>   at 
> org.apache.hadoop.hdfs.server.datanode.TestDataNodeHotSwapVolumes.testRemoveVolumeBeingWritten(TestDataNodeHotSwapVolumes.java:569)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-8991) Provide information on BPOfferService in DN JMX

2016-07-07 Thread Mingliang Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-8991?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15359854#comment-15359854
 ] 

Mingliang Liu edited comment on HDFS-8991 at 7/8/16 12:03 AM:
--

Thanks [~jnp] for the review and comment. For the mini-dfs cluster, the 
hostname will be localhost and only one entry in the HA case.

Before this patch:
- Non-HA:
{code}
{"nn1":"BP-954094909-10.22.8.141-1467417399231"}
{code}
- HA:
{code}
{"nn1":"BP-21526920-10.22.8.141-1467417356425", 
"nn2":"BP-21526920-10.22.8.141-1467417356425"}

{code}

After the patch:
- Non-HA:
{code}
{"BP-1939740699-10.22.8.141-1467416841045":{"nn1:57787":"active"}}
{code}
- HA:
{code}
{"BP-1698487347-10.22.8.141-1467417257785":{"nn1:57925":"active","nn2:57927":"standby"}}
{code}


was (Author: liuml07):
Thanks [~jnp] for the review and comment. For the mini-dfs cluster, the 
hostname will be localhost and only one entry in the HA case.

Before this patch:
- Non-HA:
{code}
{"localhost":"BP-954094909-10.22.8.141-1467417399231"}
{code}
- HA:
{code}
{"localhost":"BP-21526920-10.22.8.141-1467417356425"}
{code}

After the patch:
- Non-HA:
{code}
{"BP-1939740699-10.22.8.141-1467416841045":{"localhost:57787":"active"}}
{code}
- HA:
{code}
{"BP-1698487347-10.22.8.141-1467417257785":{"localhost:57925":"active","localhost:57927":"standby"}}
{code}

> Provide information on BPOfferService in DN JMX
> ---
>
> Key: HDFS-8991
> URL: https://issues.apache.org/jira/browse/HDFS-8991
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Reporter: Haohui Mai
>Assignee: Mingliang Liu
> Attachments: HDFS-8991.000.patch, HDFS-8991.001.patch
>
>
> In cases like HDFS-7714 where the BPOfferService thread is missing, which 
> require nontrivial effort to debug that which NN that the DN thinks it active 
> / standby.
> It would make sense to make the information more accessible through JMX or 
> Web UI.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10590) Fix TestReconstructStripedBlocks.testCountLiveReplicas test failures

2016-07-07 Thread Uma Maheswara Rao G (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366969#comment-15366969
 ] 

Uma Maheswara Rao G commented on HDFS-10590:


+1, Thanks Rakesh

> Fix TestReconstructStripedBlocks.testCountLiveReplicas test failures
> 
>
> Key: HDFS-10590
> URL: https://issues.apache.org/jira/browse/HDFS-10590
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: Rakesh R
>Assignee: Rakesh R
> Attachments: HDFS-10590-00.patch
>
>
> This jira is to fix the test case failure. Please see the below stacktrace.
> Reference : 
> [Build_15968|https://builds.apache.org/job/PreCommit-HDFS-Build/15968/testReport/junit/org.apache.hadoop.hdfs.server.namenode/TestReconstructStripedBlocks/testCountLiveReplicas/]
> {code}
> java.lang.AssertionError: null
>   at org.junit.Assert.fail(Assert.java:86)
>   at org.junit.Assert.assertTrue(Assert.java:41)
>   at org.junit.Assert.assertTrue(Assert.java:52)
>   at 
> org.apache.hadoop.hdfs.server.namenode.TestReconstructStripedBlocks.testCountLiveReplicas(TestReconstructStripedBlocks.java:324)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10477) Stop decommission a rack of DataNodes caused NameNode fail over to standby

2016-07-07 Thread yunjiong zhao (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366966#comment-15366966
 ] 

yunjiong zhao commented on HDFS-10477:
--

Those failed unit test is not related to this patch.
And there is no need to add new unit test for this patch since it's only add 
steps to release the nameSystem writeLock and then acquire the lock again.

> Stop decommission a rack of DataNodes caused NameNode fail over to standby
> --
>
> Key: HDFS-10477
> URL: https://issues.apache.org/jira/browse/HDFS-10477
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: namenode
>Affects Versions: 2.7.2
>Reporter: yunjiong zhao
>Assignee: yunjiong zhao
> Attachments: HDFS-10477.002.patch, HDFS-10477.003.patch, 
> HDFS-10477.patch
>
>
> In our cluster, when we stop decommissioning a rack which have 46 DataNodes, 
> it locked Namesystem for about 7 minutes as below log shows:
> {code}
> 2016-05-26 20:11:41,697 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.27:1004
> 2016-05-26 20:11:51,171 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 285258 over-replicated blocks on 10.142.27.27:1004 during recommissioning
> 2016-05-26 20:11:51,171 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.118:1004
> 2016-05-26 20:11:59,972 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 279923 over-replicated blocks on 10.142.27.118:1004 during recommissioning
> 2016-05-26 20:11:59,972 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.113:1004
> 2016-05-26 20:12:09,007 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 294307 over-replicated blocks on 10.142.27.113:1004 during recommissioning
> 2016-05-26 20:12:09,008 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.117:1004
> 2016-05-26 20:12:18,055 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 314381 over-replicated blocks on 10.142.27.117:1004 during recommissioning
> 2016-05-26 20:12:18,056 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.130:1004
> 2016-05-26 20:12:25,938 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 272779 over-replicated blocks on 10.142.27.130:1004 during recommissioning
> 2016-05-26 20:12:25,939 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.121:1004
> 2016-05-26 20:12:34,134 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 287248 over-replicated blocks on 10.142.27.121:1004 during recommissioning
> 2016-05-26 20:12:34,134 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.33:1004
> 2016-05-26 20:12:43,020 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 299868 over-replicated blocks on 10.142.27.33:1004 during recommissioning
> 2016-05-26 20:12:43,020 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.137:1004
> 2016-05-26 20:12:52,220 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 303914 over-replicated blocks on 10.142.27.137:1004 during recommissioning
> 2016-05-26 20:12:52,220 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.51:1004
> 2016-05-26 20:13:00,362 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 281175 over-replicated blocks on 10.142.27.51:1004 during recommissioning
> 2016-05-26 20:13:00,362 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.12:1004
> 2016-05-26 20:13:08,756 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 274880 over-replicated blocks on 10.142.27.12:1004 during recommissioning
> 2016-05-26 20:13:08,757 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.15:1004
> 2016-05-26 20:13:17,185 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 286334 over-replicated blocks on 10.142.27.15:1004 during recommissioning
> 2016-05-26 20:13:17,185 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Stop 
> Decommissioning 10.142.27.14:1004
> 2016-05-26 20:13:25,369 INFO 
> org.apache.hadoop.hdfs.server.blockmanagement.BlockManager: Invalidated 
> 280219 over-replicated blocks on 10.142.27.14:1004

[jira] [Commented] (HDFS-8940) Support for large-scale multi-tenant inotify service

2016-07-07 Thread Kai Zheng (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366960#comment-15366960
 ] 

Kai Zheng commented on HDFS-8940:
-

Hi [~mingma],

Could you help clarify if it works or not, in the way I said above that your 
Zookeeper could get triggered by Kafka instead of HDFS directly? Or simply: 
HDFS->Kafka->Zookeeper, instead of HDFS->Zookeeper. 

I'm thinking that having a Kafka expose all the NameNode activities allowing 
all kinds of possible consumers may best align all the requirements like raised 
here. My side are interested in this and hope we can sort out an approach so to 
contribute.

> Support for large-scale multi-tenant inotify service
> 
>
> Key: HDFS-8940
> URL: https://issues.apache.org/jira/browse/HDFS-8940
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Ming Ma
> Attachments: Large-Scale-Multi-Tenant-Inotify-Service.pdf
>
>
> HDFS-6634 provides the core inotify functionality. We would like to extend 
> that to provide a large-scale service that ten of thousands of clients can 
> subscribe to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-9805) TCP_NODELAY not set before SASL handshake in data transfer pipeline

2016-07-07 Thread Gary Helmling (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gary Helmling updated HDFS-9805:

Attachment: HDFS-9805.branch-2.001.patch

[~cmccabe] Here is a patch backported to branch-2.  It should apply cleanly.  
Do I need to re-open the issue to get the Yetus build to run?

> TCP_NODELAY not set before SASL handshake in data transfer pipeline
> ---
>
> Key: HDFS-9805
> URL: https://issues.apache.org/jira/browse/HDFS-9805
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: datanode
>Reporter: Gary Helmling
>Assignee: Gary Helmling
> Fix For: 3.0.0-alpha1
>
> Attachments: HDFS-9805.002.patch, HDFS-9805.003.patch, 
> HDFS-9805.004.patch, HDFS-9805.005.patch, HDFS-9805.branch-2.001.patch
>
>
> There are a few places in the DN -> DN block transfer pipeline where 
> TCP_NODELAY is not set before doing a SASL handshake:
> * in {{DataNode.DataTransfer::run()}}
> * in {{DataXceiver::replaceBlock()}}
> * in {{DataXceiver::writeBlock()}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10534) NameNode WebUI should display DataNode usage histogram

2016-07-07 Thread Zhe Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366931#comment-15366931
 ] 

Zhe Zhang commented on HDFS-10534:
--

So the v6 patch doesn't apply on current trunk. I applied {{d3.min.js}} but it 
is not formatted. Also why is the name {{d3.min}}? Thanks.

> NameNode WebUI should display DataNode usage histogram
> --
>
> Key: HDFS-10534
> URL: https://issues.apache.org/jira/browse/HDFS-10534
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: namenode, ui
>Reporter: Zhe Zhang
>Assignee: Kai Sasaki
> Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, 
> HDFS-10534.03.patch, HDFS-10534.04.patch, HDFS-10534.05.patch, 
> HDFS-10534.06.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png, Screen Shot 
> 2016-07-07 at 23.29.14.png
>
>
> In addition of *Min/Median/Max*, another meaningful metric for cluster 
> balance is DN usage in histogram style.
> Since NN already has provided necessary information to calculate histogram of 
> DN usage, it can be done in JS side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-07-07 Thread Eric Badger (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Badger updated HDFS-5805:
--
Attachment: HDFS-5805.001.patch

Attaching a patch that adds a loop of mkdir and delete calls, so that the edit 
log time will never be so low that it is truncated to 0. Without this patch, 
the test fails about 1/3 of the time for me locally (with an SSD). With the 
patch, it failed 0 times out of 50 attempts. 

> TestCheckpoint.testCheckpoint fails intermittently on branch2
> -
>
> Key: HDFS-5805
> URL: https://issues.apache.org/jira/browse/HDFS-5805
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Mit Desai
>Assignee: Eric Badger
> Attachments: HDFS-5805.001.patch
>
>
> {noformat}
> java.lang.AssertionError: Bad value for metric GetEditAvgTime
> Expected: gt(0.0)
>  got: <0.0>
>   at org.junit.Assert.assertThat(Assert.java:780)
>   at 
> org.apache.hadoop.test.MetricsAsserts.assertGaugeGt(MetricsAsserts.java:341)
>   at 
> org.apache.hadoop.hdfs.server.namenode.TestCheckpoint.testCheckpoint(TestCheckpoint.java:1070)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10534) NameNode WebUI should display DataNode usage histogram

2016-07-07 Thread Zhe Zhang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366927#comment-15366927
 ] 

Zhe Zhang commented on HDFS-10534:
--

Thanks [~lewuathe]! Screenshot looks very nice.

On the first glance the js file added in the patch is very large. Is it 
necessary? I haven't actually applied and reviewed it, just a quick comment. 

> NameNode WebUI should display DataNode usage histogram
> --
>
> Key: HDFS-10534
> URL: https://issues.apache.org/jira/browse/HDFS-10534
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: namenode, ui
>Reporter: Zhe Zhang
>Assignee: Kai Sasaki
> Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, 
> HDFS-10534.03.patch, HDFS-10534.04.patch, HDFS-10534.05.patch, 
> HDFS-10534.06.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png, Screen Shot 
> 2016-07-07 at 23.29.14.png
>
>
> In addition of *Min/Median/Max*, another meaningful metric for cluster 
> balance is DN usage in histogram style.
> Since NN already has provided necessary information to calculate histogram of 
> DN usage, it can be done in JS side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-07-07 Thread Eric Badger (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Badger updated HDFS-5805:
--
Status: Patch Available  (was: Reopened)

> TestCheckpoint.testCheckpoint fails intermittently on branch2
> -
>
> Key: HDFS-5805
> URL: https://issues.apache.org/jira/browse/HDFS-5805
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Mit Desai
>Assignee: Eric Badger
> Attachments: HDFS-5805.001.patch
>
>
> {noformat}
> java.lang.AssertionError: Bad value for metric GetEditAvgTime
> Expected: gt(0.0)
>  got: <0.0>
>   at org.junit.Assert.assertThat(Assert.java:780)
>   at 
> org.apache.hadoop.test.MetricsAsserts.assertGaugeGt(MetricsAsserts.java:341)
>   at 
> org.apache.hadoop.hdfs.server.namenode.TestCheckpoint.testCheckpoint(TestCheckpoint.java:1070)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Assigned] (HDFS-5805) TestCheckpoint.testCheckpoint fails intermittently on branch2

2016-07-07 Thread Eric Badger (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-5805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Badger reassigned HDFS-5805:
-

Assignee: Eric Badger  (was: Mit Desai)

> TestCheckpoint.testCheckpoint fails intermittently on branch2
> -
>
> Key: HDFS-5805
> URL: https://issues.apache.org/jira/browse/HDFS-5805
> Project: Hadoop HDFS
>  Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Mit Desai
>Assignee: Eric Badger
>
> {noformat}
> java.lang.AssertionError: Bad value for metric GetEditAvgTime
> Expected: gt(0.0)
>  got: <0.0>
>   at org.junit.Assert.assertThat(Assert.java:780)
>   at 
> org.apache.hadoop.test.MetricsAsserts.assertGaugeGt(MetricsAsserts.java:341)
>   at 
> org.apache.hadoop.hdfs.server.namenode.TestCheckpoint.testCheckpoint(TestCheckpoint.java:1070)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Larry McCay (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366916#comment-15366916
 ] 

Larry McCay commented on HDFS-10579:


[~anu] - This looks good.
I will review the new patches when they arrive as well.

Thanks for adding this!

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Comment Edited] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Anu Engineer (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366858#comment-15366858
 ] 

Anu Engineer edited comment on HDFS-10579 at 7/7/16 10:07 PM:
--

[~haibochen] Thanks for the comments

bq.  xFrameOption and xFrameOptionIsEnabled (of HttpServer2) are declared as 
static but used as instance variables. Can you make them non-static? Of course, 
QuotingInputFilter has to be non-static to access them

Just wanted to let you know that I will make this change and post that patch to 
the new Hadoop Common Jira that I will be creating based on your suggestion. I 
will tag you on that Jira so that you can look at the changes. I should be able 
to post a patch by EOD. btw, looks like YARN addressed this issue in this 
patch. https://issues.apache.org/jira/browse/YARN-5076


was (Author: anu):
[~haibochen] Thanks for the comments

bq.  xFrameOption and xFrameOptionIsEnabled (of HttpServer2) are declared as 
static but used as instance variables. Can you make them non-static? Of course, 
QuotingInputFilter has to be non-static to access them

Just wanted to let you know that I will make this change and post that patch to 
the new Hadoop Common Jira that I will be creating based on your suggestion. I 
will tag on that Jira so that you can look at the changes. I should be able to 
post a patch by EOD. btw, looks like YARN addressed this issue in this patch. 
https://issues.apache.org/jira/browse/YARN-5076

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Anu Engineer (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366858#comment-15366858
 ] 

Anu Engineer commented on HDFS-10579:
-

[~haibochen] Thanks for the comments

bq.  xFrameOption and xFrameOptionIsEnabled (of HttpServer2) are declared as 
static but used as instance variables. Can you make them non-static? Of course, 
QuotingInputFilter has to be non-static to access them

Just wanted to let you know that I will make this change and post that patch to 
the new Hadoop Common Jira that I will be creating based on your suggestion. I 
will tag on that Jira so that you can look at the changes. I should be able to 
post a patch by EOD. btw, looks like YARN addressed this issue in this patch. 
https://issues.apache.org/jira/browse/YARN-5076

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2016-07-07 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366843#comment-15366843
 ] 

Hadoop QA commented on HDFS-10601:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
16s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 26s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch 
generated 1 new + 218 unchanged - 1 fixed = 219 total (was 219) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 61m 18s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 81m 36s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.cli.TestHDFSCLI |
|   | hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits |
|   | hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816688/HDFS-10601.002.patch |
| JIRA Issue | HDFS-10601 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux c5d057fe1f21 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / d1626a7 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16002/artifact/patchprocess/diff-checkstyle-hadoop-hdfs-project_hadoop-hdfs.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16002/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16002/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/16002/console |
| Powered by | Apache Yetus 0.4.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Improve log message to include hostname when the NameNode is in

[jira] [Commented] (HDFS-10592) Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning

2016-07-07 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366788#comment-15366788
 ] 

Hudson commented on HDFS-10592:
---

SUCCESS: Integrated in Hadoop-trunk-Commit #10062 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/10062/])
HDFS-10592:Fix intermittent test failure of (uma.gangumalla: rev 
ab092c56c251f27a9426e3c69a784417fdf9a26e)
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestNameNodeResourceChecker.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/tools/TestDFSZKFailoverController.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/ha/TestNNHealthCheck.java
* 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/MockNameNodeResourceChecker.java


> Fix intermittent test failure of 
> TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning
> --
>
> Key: HDFS-10592
> URL: https://issues.apache.org/jira/browse/HDFS-10592
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: 2.8.0
>
> Attachments: HDFS-10592-00.patch, HDFS-10592-01.patch
>
>
> This jira is to fix the 
> {{TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning}} 
> test case failure.
> Reference 
> [Build_15973|https://builds.apache.org/job/PreCommit-HDFS-Build/15973/testReport/junit/org.apache.hadoop.hdfs.server.namenode/TestNameNodeResourceChecker/testCheckThatNameNodeResourceMonitorIsRunning/]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10592) Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning

2016-07-07 Thread Uma Maheswara Rao G (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HDFS-10592:
---
  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

I have just committed this to trunk and branch-2, 2.8. Thanks

> Fix intermittent test failure of 
> TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning
> --
>
> Key: HDFS-10592
> URL: https://issues.apache.org/jira/browse/HDFS-10592
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: 2.8.0
>
> Attachments: HDFS-10592-00.patch, HDFS-10592-01.patch
>
>
> This jira is to fix the 
> {{TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning}} 
> test case failure.
> Reference 
> [Build_15973|https://builds.apache.org/job/PreCommit-HDFS-Build/15973/testReport/junit/org.apache.hadoop.hdfs.server.namenode/TestNameNodeResourceChecker/testCheckThatNameNodeResourceMonitorIsRunning/]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10592) Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning

2016-07-07 Thread Uma Maheswara Rao G (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Uma Maheswara Rao G updated HDFS-10592:
---
Component/s: test

> Fix intermittent test failure of 
> TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning
> --
>
> Key: HDFS-10592
> URL: https://issues.apache.org/jira/browse/HDFS-10592
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: 2.8.0
>
> Attachments: HDFS-10592-00.patch, HDFS-10592-01.patch
>
>
> This jira is to fix the 
> {{TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning}} 
> test case failure.
> Reference 
> [Build_15973|https://builds.apache.org/job/PreCommit-HDFS-Build/15973/testReport/junit/org.apache.hadoop.hdfs.server.namenode/TestNameNodeResourceChecker/testCheckThatNameNodeResourceMonitorIsRunning/]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Haibo Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366764#comment-15366764
 ] 

Haibo Chen commented on HDFS-10579:
---

Thanks [~anu] a lot for working on this! I totally did not notice compatibility 
issues on branch-2. A few comments. 
1)  xFrameOption and xFrameOptionIsEnabled (of HttpServer2) are declared as 
static but used as instance variables. Can you make them non-static? Of course, 
QuotingInputFilter has to be non-static to access them.
2) testHttpResonseContainsXFrameOptions, testHttpResonseContainsDeny and 
testHttpResonseContainsAllowFrom are the same except the x-frame-option config. 
You could have a common method that takes x-frame-option as a parameter and 
does the verification. Then the three test methods can simply call that method 
with different x-frame-option.
3) The patch touches both HttpServer2 which is in COMMON, and HDFS servers. Can 
you create a parent jira against COMMON to make HttpServer2 changes, then 
create a sub task against HDFS of that to make HDFS changes? Other components 
also uses HttpServer2, such as MR. If needed, we could add more subtasks for 
each of the components.

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2016-07-07 Thread Kuhu Shukla (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kuhu Shukla updated HDFS-10601:
---
Attachment: HDFS-10601.002.patch

Updated patch to pick up the hostname from NameNode static method. Corrected 
checkstyle issues except the one regarding method length.

> Improve log message to include hostname when the NameNode is in safemode
> 
>
> Key: HDFS-10601
> URL: https://issues.apache.org/jira/browse/HDFS-10601
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
>Priority: Minor
> Attachments: HDFS-10601.001.patch, HDFS-10601.002.patch
>
>
> When remote NN operations are involved, it would be nice to have the Namenode 
> hostname in safemode notification log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10592) Fix intermittent test failure of TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning

2016-07-07 Thread Uma Maheswara Rao G (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366704#comment-15366704
 ] 

Uma Maheswara Rao G commented on HDFS-10592:


Thanks Rakesh for working on this.
+1 for the patch. Will commit this.

> Fix intermittent test failure of 
> TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning
> --
>
> Key: HDFS-10592
> URL: https://issues.apache.org/jira/browse/HDFS-10592
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: Rakesh R
>Assignee: Rakesh R
> Fix For: 2.8.0
>
> Attachments: HDFS-10592-00.patch, HDFS-10592-01.patch
>
>
> This jira is to fix the 
> {{TestNameNodeResourceChecker#testCheckThatNameNodeResourceMonitorIsRunning}} 
> test case failure.
> Reference 
> [Build_15973|https://builds.apache.org/job/PreCommit-HDFS-Build/15973/testReport/junit/org.apache.hadoop.hdfs.server.namenode/TestNameNodeResourceChecker/testCheckThatNameNodeResourceMonitorIsRunning/]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Anu Engineer (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366700#comment-15366700
 ] 

Anu Engineer commented on HDFS-10579:
-

Test Failures are not related to this patch.

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2016-07-07 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366603#comment-15366603
 ] 

Hadoop QA commented on HDFS-10601:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
49s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
45s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
0m 26s{color} | {color:orange} hadoop-hdfs-project/hadoop-hdfs: The patch 
generated 2 new + 218 unchanged - 1 fixed = 220 total (was 219) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 60m 16s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
20s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 51s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestDFSFinalize |
|   | hadoop.hdfs.TestDFSStartupVersions |
|   | hadoop.hdfs.server.mover.TestStorageMover |
|   | hadoop.hdfs.TestFileChecksum |
|   | hadoop.hdfs.server.namenode.TestCommitBlockSynchronization |
|   | hadoop.hdfs.server.namenode.TestFsLimits |
|   | hadoop.cli.TestHDFSCLI |
|   | hadoop.hdfs.server.namenode.TestNNThroughputBenchmark |
|   | hadoop.hdfs.server.namenode.TestSecurityTokenEditLog |
|   | hadoop.hdfs.TestDFSUpgrade |
|   | hadoop.hdfs.server.namenode.TestFSNamesystem |
|   | hadoop.hdfs.TestDFSStorageStateRecovery |
|   | hadoop.hdfs.server.namenode.TestGetBlockLocations |
|   | hadoop.hdfs.TestDFSRollback |
|   | hadoop.hdfs.server.namenode.TestClusterId |
|   | hadoop.hdfs.server.namenode.TestCreateEditsLog |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816670/HDFS-10601.001.patch |
| JIRA Issue | HDFS-10601 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 6f161f4595a7 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / b8f93cd |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |

[jira] [Commented] (HDFS-10488) Update WebHDFS documentation regarding CREATE and MKDIR default permissions

2016-07-07 Thread Hudson (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366504#comment-15366504
 ] 

Hudson commented on HDFS-10488:
---

SUCCESS: Integrated in Hadoop-trunk-Commit #10059 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/10059/])
HDFS-10488. Update WebHDFS documentation regarding CREATE and MKDIR (wang: rev 
b8f93cd2750cde31d3e80c1aecd426e0cea89ef2)
* hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/WebHDFS.md


> Update WebHDFS documentation regarding CREATE and MKDIR default permissions
> ---
>
> Key: HDFS-10488
> URL: https://issues.apache.org/jira/browse/HDFS-10488
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: documentation, webhdfs
>Affects Versions: 2.6.0
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 2.8.0, 2.7.3, 3.0.0-alpha1
>
> Attachments: HDFS-10488.002.patch, HDFS-10488.003.patch, 
> HDFS-10488.005.patch, HDFS-10488.006.patch, HDFS-10488.patch
>
>
> WebHDFS methods for creating file/directories were always creating it with 
> 755 permissions as default for both files and directories. 
> The configured *fs.permissions.umask-mode* is intentionally ignored.
> This jira is to update the Documentation properly, explaining *umask* is not 
> applied when using WebHDFS related methods.
> HDFS-6434 has also modified the default permissions for files, which is now 
> *644*. This will also be updated on the current documentation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10488) Update WebHDFS documentation regarding CREATE and MKDIR default permissions

2016-07-07 Thread Andrew Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang updated HDFS-10488:
---
Issue Type: Improvement  (was: Bug)

> Update WebHDFS documentation regarding CREATE and MKDIR default permissions
> ---
>
> Key: HDFS-10488
> URL: https://issues.apache.org/jira/browse/HDFS-10488
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: documentation, webhdfs
>Affects Versions: 2.6.0
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 2.8.0, 2.7.3, 3.0.0-alpha1
>
> Attachments: HDFS-10488.002.patch, HDFS-10488.003.patch, 
> HDFS-10488.005.patch, HDFS-10488.006.patch, HDFS-10488.patch
>
>
> WebHDFS methods for creating file/directories were always creating it with 
> 755 permissions as default for both files and directories. 
> The configured *fs.permissions.umask-mode* is intentionally ignored.
> This jira is to update the Documentation properly, explaining *umask* is not 
> applied when using WebHDFS related methods.
> HDFS-6434 has also modified the default permissions for files, which is now 
> *644*. This will also be updated on the current documentation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Resolved] (HDFS-10488) Update WebHDFS documentation regarding CREATE and MKDIR default permissions

2016-07-07 Thread Andrew Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang resolved HDFS-10488.

   Resolution: Fixed
Fix Version/s: 3.0.0-alpha1
   2.7.3
   2.8.0

Committed, thanks Wellington for the patch, and everyone for the reviews!

> Update WebHDFS documentation regarding CREATE and MKDIR default permissions
> ---
>
> Key: HDFS-10488
> URL: https://issues.apache.org/jira/browse/HDFS-10488
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: documentation, webhdfs
>Affects Versions: 2.6.0
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 2.8.0, 2.7.3, 3.0.0-alpha1
>
> Attachments: HDFS-10488.002.patch, HDFS-10488.003.patch, 
> HDFS-10488.005.patch, HDFS-10488.006.patch, HDFS-10488.patch
>
>
> WebHDFS methods for creating file/directories were always creating it with 
> 755 permissions as default for both files and directories. 
> The configured *fs.permissions.umask-mode* is intentionally ignored.
> This jira is to update the Documentation properly, explaining *umask* is not 
> applied when using WebHDFS related methods.
> HDFS-6434 has also modified the default permissions for files, which is now 
> *644*. This will also be updated on the current documentation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10488) Update WebHDFS documentation regarding CREATE and MKDIR default permissions

2016-07-07 Thread Andrew Wang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang updated HDFS-10488:
---
Summary: Update WebHDFS documentation regarding CREATE and MKDIR default 
permissions  (was: Update Documentation for WebHDFS CREATE and MKDIR methods to 
reflect the default permissions changes applied on HDFS-6434)

> Update WebHDFS documentation regarding CREATE and MKDIR default permissions
> ---
>
> Key: HDFS-10488
> URL: https://issues.apache.org/jira/browse/HDFS-10488
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: documentation, webhdfs
>Affects Versions: 2.6.0
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Attachments: HDFS-10488.002.patch, HDFS-10488.003.patch, 
> HDFS-10488.005.patch, HDFS-10488.006.patch, HDFS-10488.patch
>
>
> WebHDFS methods for creating file/directories were always creating it with 
> 755 permissions as default for both files and directories. 
> The configured *fs.permissions.umask-mode* is intentionally ignored.
> This jira is to update the Documentation properly, explaining *umask* is not 
> applied when using WebHDFS related methods.
> HDFS-6434 has also modified the default permissions for files, which is now 
> *644*. This will also be updated on the current documentation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10488) Update Documentation for WebHDFS CREATE and MKDIR methods to reflect the default permissions changes applied on HDFS-6434

2016-07-07 Thread Andrew Wang (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366451#comment-15366451
 ] 

Andrew Wang commented on HDFS-10488:


About to commit the respective patches, thanks all.

> Update Documentation for WebHDFS CREATE and MKDIR methods to reflect the 
> default permissions changes applied on HDFS-6434
> -
>
> Key: HDFS-10488
> URL: https://issues.apache.org/jira/browse/HDFS-10488
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: documentation, webhdfs
>Affects Versions: 2.6.0
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Attachments: HDFS-10488.002.patch, HDFS-10488.003.patch, 
> HDFS-10488.005.patch, HDFS-10488.006.patch, HDFS-10488.patch
>
>
> WebHDFS methods for creating file/directories were always creating it with 
> 755 permissions as default for both files and directories. 
> The configured *fs.permissions.umask-mode* is intentionally ignored.
> This jira is to update the Documentation properly, explaining *umask* is not 
> applied when using WebHDFS related methods.
> HDFS-6434 has also modified the default permissions for files, which is now 
> *644*. This will also be updated on the current documentation.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2016-07-07 Thread Kuhu Shukla (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kuhu Shukla updated HDFS-10601:
---
Status: Patch Available  (was: Open)

> Improve log message to include hostname when the NameNode is in safemode
> 
>
> Key: HDFS-10601
> URL: https://issues.apache.org/jira/browse/HDFS-10601
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
>Priority: Minor
> Attachments: HDFS-10601.001.patch
>
>
> When remote NN operations are involved, it would be nice to have the Namenode 
> hostname in safemode notification log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2016-07-07 Thread Kuhu Shukla (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kuhu Shukla updated HDFS-10601:
---
Attachment: HDFS-10601.001.patch

Attaching initial patch.

> Improve log message to include hostname when the NameNode is in safemode
> 
>
> Key: HDFS-10601
> URL: https://issues.apache.org/jira/browse/HDFS-10601
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Kuhu Shukla
>Assignee: Kuhu Shukla
>Priority: Minor
> Attachments: HDFS-10601.001.patch
>
>
> When remote NN operations are involved, it would be nice to have the Namenode 
> hostname in safemode notification log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Created] (HDFS-10601) Improve log message to include hostname when the NameNode is in safemode

2016-07-07 Thread Kuhu Shukla (JIRA)

Kuhu Shukla created HDFS-10601:
--

 Summary: Improve log message to include hostname when the NameNode 
is in safemode
 Key: HDFS-10601
 URL: https://issues.apache.org/jira/browse/HDFS-10601
 Project: Hadoop HDFS
  Issue Type: Improvement
Reporter: Kuhu Shukla
Assignee: Kuhu Shukla
Priority: Minor


When remote NN operations are involved, it would be nice to have the Namenode 
hostname in safemode notification log.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10336) TestBalancer failing intermittently because of not reseting UserGroupInformation completely

2016-07-07 Thread Xiao Chen (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366411#comment-15366411
 ] 

Xiao Chen commented on HDFS-10336:
--

Thanks [~linyiqun].
Please update the timeout of {{testUnknownDatanodeSimple}} to be the same, 
since it's calling the same underlying method.

Also, looking closer, {{testBalancerWithKeytabs}} has a 5 minute timeout, not 
30s. Are you sure the test passes after bumping this to 10 mins? Would prefer 
to allow the test to pass sooner if possible. I'm okay to defer this to a 
separate jira too.

> TestBalancer failing intermittently because of not reseting 
> UserGroupInformation completely
> ---
>
> Key: HDFS-10336
> URL: https://issues.apache.org/jira/browse/HDFS-10336
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: test
>Affects Versions: 3.0.0-alpha1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HDFS-10336.001.patch, HDFS-10336.002.patch
>
>
> The unit test {{TestBalancer}} failed sometimes. 
> I looked for the reason. I found two main reasons causing this.
> * 1st. The test {{TestBalancer#testBalancerWithKeytabs}} executed timeout.
> {code}
> org.apache.hadoop.hdfs.server.balancer.TestBalancer
> testBalancerWithKeytabs(org.apache.hadoop.hdfs.server.balancer.TestBalancer)  
> Time elapsed: 300.41 sec  <<< ERROR!
> java.lang.Exception: test timed out after 30 milliseconds
>   at java.lang.Thread.sleep(Native Method)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.waitForMoveCompletion(Dispatcher.java:1122)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchBlockMoves(Dispatcher.java:1096)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Dispatcher.dispatchAndCheckContinue(Dispatcher.java:1060)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.runOneIteration(Balancer.java:635)
>   at 
> org.apache.hadoop.hdfs.server.balancer.Balancer.run(Balancer.java:689)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.testUnknownDatanode(TestBalancer.java:1098)
>   at 
> org.apache.hadoop.hdfs.server.balancer.TestBalancer.access$000(TestBalancer.java:125)
> {code}
> * 2nd. The test {{TestBalancer#testBalancerWithKeytabs}} reset the {{UGI}} 
> not completely sometimes in the finally block. And this affected the other 
> unit tests threw {{IOException}}, like this:
> {code}
> testBalancerWithNonZeroThreadsForMove(org.apache.hadoop.hdfs.server.balancer.TestBalancer)
>   Time elapsed: 0 sec  <<< ERROR!
> java.io.IOException: Running in secure mode, but config doesn't have a keytab
>   at org.apache.hadoop.security.SecurityUtil.login(SecurityUtil.java:300)
> {code}
> And there were not only one test will be affected by this. We should add a 
> line to do before doing reset {{UGI}} operation and can avoid the potenial 
> exception happens.
> {code}
> UserGroupInformation.reset();
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-8940) Support for large-scale multi-tenant inotify service

2016-07-07 Thread Ming Ma (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366286#comment-15366286
 ] 

Ming Ma commented on HDFS-8940:
---

Thanks [~surendrasingh] and [~drankye]. One of the original motivations is to 
support large number of clients. Zookeeper support separation of quorum write 
members from the followers, thus allow you to add more machines to scale out 
read volume. JournalNode doesn't seem to have such functionality.

> Support for large-scale multi-tenant inotify service
> 
>
> Key: HDFS-8940
> URL: https://issues.apache.org/jira/browse/HDFS-8940
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Ming Ma
> Attachments: Large-Scale-Multi-Tenant-Inotify-Service.pdf
>
>
> HDFS-6634 provides the core inotify functionality. We would like to extend 
> that to provide a large-scale service that ten of thousands of clients can 
> subscribe to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10534) NameNode WebUI should display DataNode usage histogram

2016-07-07 Thread Kai Sasaki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kai Sasaki updated HDFS-10534:
--
Status: Patch Available  (was: Reopened)

> NameNode WebUI should display DataNode usage histogram
> --
>
> Key: HDFS-10534
> URL: https://issues.apache.org/jira/browse/HDFS-10534
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: namenode, ui
>Reporter: Zhe Zhang
>Assignee: Kai Sasaki
> Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, 
> HDFS-10534.03.patch, HDFS-10534.04.patch, HDFS-10534.05.patch, 
> HDFS-10534.06.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png, Screen Shot 
> 2016-07-07 at 23.29.14.png
>
>
> In addition of *Min/Median/Max*, another meaningful metric for cluster 
> balance is DN usage in histogram style.
> Since NN already has provided necessary information to calculate histogram of 
> DN usage, it can be done in JS side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10534) NameNode WebUI should display DataNode usage histogram

2016-07-07 Thread Kai Sasaki (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366212#comment-15366212
 ] 

Kai Sasaki commented on HDFS-10534:
---

[~zhz] I implemented node usage histogram UI and attached screen shot of the 
view.
Could you give me some feedback if you are available? Thanks.

> NameNode WebUI should display DataNode usage histogram
> --
>
> Key: HDFS-10534
> URL: https://issues.apache.org/jira/browse/HDFS-10534
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: namenode, ui
>Reporter: Zhe Zhang
>Assignee: Kai Sasaki
> Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, 
> HDFS-10534.03.patch, HDFS-10534.04.patch, HDFS-10534.05.patch, 
> HDFS-10534.06.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png, Screen Shot 
> 2016-07-07 at 23.29.14.png
>
>
> In addition of *Min/Median/Max*, another meaningful metric for cluster 
> balance is DN usage in histogram style.
> Since NN already has provided necessary information to calculate histogram of 
> DN usage, it can be done in JS side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10534) NameNode WebUI should display DataNode usage histogram

2016-07-07 Thread Kai Sasaki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kai Sasaki updated HDFS-10534:
--
Attachment: Screen Shot 2016-07-07 at 23.29.14.png

> NameNode WebUI should display DataNode usage histogram
> --
>
> Key: HDFS-10534
> URL: https://issues.apache.org/jira/browse/HDFS-10534
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: namenode, ui
>Reporter: Zhe Zhang
>Assignee: Kai Sasaki
> Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, 
> HDFS-10534.03.patch, HDFS-10534.04.patch, HDFS-10534.05.patch, 
> HDFS-10534.06.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png, Screen Shot 
> 2016-07-07 at 23.29.14.png
>
>
> In addition of *Min/Median/Max*, another meaningful metric for cluster 
> balance is DN usage in histogram style.
> Since NN already has provided necessary information to calculate histogram of 
> DN usage, it can be done in JS side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10534) NameNode WebUI should display DataNode usage histogram

2016-07-07 Thread Kai Sasaki (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kai Sasaki updated HDFS-10534:
--
Attachment: HDFS-10534.06.patch

> NameNode WebUI should display DataNode usage histogram
> --
>
> Key: HDFS-10534
> URL: https://issues.apache.org/jira/browse/HDFS-10534
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: namenode, ui
>Reporter: Zhe Zhang
>Assignee: Kai Sasaki
> Attachments: HDFS-10534.01.patch, HDFS-10534.02.patch, 
> HDFS-10534.03.patch, HDFS-10534.04.patch, HDFS-10534.05.patch, 
> HDFS-10534.06.patch, Screen Shot 2016-06-23 at 6.25.50 AM.png, Screen Shot 
> 2016-07-07 at 23.29.14.png
>
>
> In addition of *Min/Median/Max*, another meaningful metric for cluster 
> balance is DN usage in histogram style.
> Since NN already has provided necessary information to calculate histogram of 
> DN usage, it can be done in JS side.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-8940) Support for large-scale multi-tenant inotify service

2016-07-07 Thread Kai Zheng (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15366145#comment-15366145
 ] 

Kai Zheng commented on HDFS-8940:
-

It looks to me the Kafka approach would be much more useful, because it can 
integrate with many popular existing consumer systems at the same time. To 
solve the issue [~mingma] mentioned before where a Zookeeper cluster may be 
better, maybe Zookeeper can be populated with the events as a Kafka consumer to 
support lots of client users?

> Support for large-scale multi-tenant inotify service
> 
>
> Key: HDFS-8940
> URL: https://issues.apache.org/jira/browse/HDFS-8940
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Ming Ma
> Attachments: Large-Scale-Multi-Tenant-Inotify-Service.pdf
>
>
> HDFS-6634 provides the core inotify functionality. We would like to extend 
> that to provide a large-scale service that ten of thousands of clients can 
> subscribe to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10594) HDFS-4949 should support recursive cache directives

2016-07-07 Thread Yiqun Lin (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365856#comment-15365856
 ] 

Yiqun Lin commented on HDFS-10594:
--

Thanks [~cnauroth] and [~andrew.wang] for the comments. I also agree with the 
Chris. I will post a initial patch recently.

> HDFS-4949 should support recursive cache directives
> ---
>
> Key: HDFS-10594
> URL: https://issues.apache.org/jira/browse/HDFS-10594
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: caching
>Affects Versions: 2.7.1
>Reporter: Yiqun Lin
>Assignee: Yiqun Lin
> Attachments: HDFS-10594.001.patch
>
>
> In {{CacheReplicationMonitor#rescanCacheDirectives}}, it should recursively 
> rescan the path when the inode of the path is a directory. In these code:
> {code}
> } else if (node.isDirectory()) {
> INodeDirectory dir = node.asDirectory();
> ReadOnlyList children = dir
> .getChildrenList(Snapshot.CURRENT_STATE_ID);
> for (INode child : children) {
>   if (child.isFile()) {
> rescanFile(directive, child.asFile());
>   }
> }
>}
> {code}
> If we did the this logic, it means that some inode files will be ignored when 
> the child inode is also a directory and there are some other child inode file 
> in it. Finally the child's child file which belong to this path will not be 
> cached.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Hadoop QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365845#comment-15365845
 ] 

Hadoop QA commented on HDFS-10579:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
28s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 4 new or modified test 
files. {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
31s{color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red}  2m 
11s{color} | {color:red} hadoop-common in trunk failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m 
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
50s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
12s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  7m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  7m  
2s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}  
1m 26s{color} | {color:orange} root: The patch generated 1 new + 523 unchanged 
- 0 fixed = 524 total (was 523) {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red}  2m 
16s{color} | {color:red} hadoop-common in the patch failed. {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green}  0m 
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}  9m 
16s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 37s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}135m 34s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency |
| Timed out junit tests | org.apache.hadoop.hdfs.TestLeaseRecovery2 |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:9560f25 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12816575/HDFS-10579.002.patch |
| JIRA Issue | HDFS-10579 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  xml  |
| uname | Linux cf5b006b3441 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / a3f93be |
| Default Java | 1.8.0_91 |
| mvnsite | 
https://builds.apache.org/job/PreCommit-HDFS-Build/15999/artifact/patchprocess/branch-mvnsite-hadoop-common-project_hadoop-common.txt
 |
| findbugs | v3.0.0 |
| checkstyle |

[jira] [Commented] (HDFS-8940) Support for large-scale multi-tenant inotify service

2016-07-07 Thread Surendra Singh Lilhore (JIRA)


[ 
https://issues.apache.org/jira/browse/HDFS-8940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15365824#comment-15365824
 ] 

Surendra Singh Lilhore commented on HDFS-8940:
--

 Hi All,

 Why we will not use JournalNode as a {{Inotify Service}} instead of adding one 
new service ?
 
 *One suggestion from my side :*
 
 We can use JournalNode as {{Inotify Service}}. In HDFS cluster we can install 
multiple Journal Node, in that we can choose one Journal Node for Inotify 
service. 

 1. We can configure one JounalNode as Inotify service. 

{noformat}
 
dfs.inotify.service
true
 
{noformat}

2. Client will connect with this JournalNode and pull the events. If client is 
connecting with some other JournalNode which is not supporting Inotify, then it 
will throw {{NotSupportInotifyException}}. 


3. SNN will not read edits from the InotifyService Journal Node. For this we 
can configure {{dfs.namenode.shared.edits.dir}} like this in NN.
{noformat}
  
dfs.namenode.shared.edits.dir

qjournal://x.x.x.x:8485;y.y.y.y:8485;z.z.z.z:8485;[Inotify]s.s.s.s:8485/cluster
  
{noformat}
 
Here {{\[Inotify\]}} indicate it is only for write not for read, so SNN will 
not try to read edits from this node.

> Support for large-scale multi-tenant inotify service
> 
>
> Key: HDFS-8940
> URL: https://issues.apache.org/jira/browse/HDFS-8940
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: Ming Ma
> Attachments: Large-Scale-Multi-Tenant-Inotify-Service.pdf
>
>
> HDFS-6634 provides the core inotify functionality. We would like to extend 
> that to provide a large-scale service that ten of thousands of clients can 
> subscribe to.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10579) HDFS web interfaces lack configs for X-FRAME-OPTIONS protection

2016-07-07 Thread Anu Engineer (JIRA)


 [ 
https://issues.apache.org/jira/browse/HDFS-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anu Engineer updated HDFS-10579:

Attachment: HDFS-10579.002.patch

Fix checkstyle warnings, test failures are not related to this patch.

> HDFS web interfaces lack configs for X-FRAME-OPTIONS protection
> ---
>
> Key: HDFS-10579
> URL: https://issues.apache.org/jira/browse/HDFS-10579
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 3.0.0-alpha1
>Reporter: Anu Engineer
>Assignee: Anu Engineer
> Fix For: 2.9.0
>
> Attachments: HDFS-10579.001.patch, HDFS-10579.002.patch
>
>
> This JIRA proposes to extend the work done in HADOOP-12964 and enable a 
> configuration value that enables or disables that option. This JIRA will also 
> add an ability to pick the right x-frame-option, since right now it looks 
> like we have hardcoded that to SAMEORIGIN.
> This allows HDFS to remain backward compatible as required by the branch-2.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

52 matches

Mail list logo