date:20190119

[jira] [Updated] (HDFS-14217) namenode and datanode are not starting

2019-01-19 Thread saichanda (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

saichanda updated HDFS-14217:
-
Priority: Trivial  (was: Major)

> namenode and datanode are not starting
> --
>
> Key: HDFS-14217
> URL: https://issues.apache.org/jira/browse/HDFS-14217
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, namenode
>Affects Versions: 2.9.2
>Reporter: saichanda
>Priority: Trivial
> Attachments: hadoop-saichanda-namenode-saichanda-OptiPlex-9020.log
>
>
> I am new to hadoop ecosystem. I am setting up a cluster with 4 nodes, one 
> master and 3 slave nodes. I have done the ground work of updating all xml 
> files in hadoop/etc/hadoop folder. I saved the slaves file. I formatted the 
> namenode. I then started the cluster with command, sbin/start-dfs.sh. I get 
> to see the lines:
> 'starting namenode on localhost starting datanode on slave1...starting 
> datanode on slave2...starting datanode on slave3... starting secondary 
> namenode. But when I run jps command in the terminal of masternode, I see 
> only Jps and Secondary Namenode. When I run jps command on the slave nodes, I 
> see only Jps, no datanode running on the slaves. What should I write in my 
> hdfs-site.xml? Currently I set the path for namenode and datanode like... 
> file:/home/user/hadoop_store/hdfs/namenode<\value> and similarly for 
> datanode as well. In the log file for namenode, I see the line saying 
> "Inconsistent state: storage directory doesn't exist or not accessible for 
> the path /home/user/hadoop_store/hdfs/namenode.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Created] (HDFS-14217) namenode and datanode are not starting

2019-01-19 Thread saichanda (JIRA)

saichanda created HDFS-14217:


 Summary: namenode and datanode are not starting
 Key: HDFS-14217
 URL: https://issues.apache.org/jira/browse/HDFS-14217
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: datanode, namenode
Affects Versions: 2.9.2
Reporter: saichanda
 Attachments: hadoop-saichanda-namenode-saichanda-OptiPlex-9020.log

I am new to hadoop ecosystem. I am setting up a cluster with 4 nodes, one 
master and 3 slave nodes. I have done the ground work of updating all xml files 
in hadoop/etc/hadoop folder. I saved the slaves file. I formatted the namenode. 
I then started the cluster with command, sbin/start-dfs.sh. I get to see the 
lines:

'starting namenode on localhost starting datanode on slave1...starting 
datanode on slave2...starting datanode on slave3... starting secondary 
namenode. But when I run jps command in the terminal of masternode, I see 
only Jps and Secondary Namenode. When I run jps command on the slave nodes, I 
see only Jps, no datanode running on the slaves. What should I write in my 
hdfs-site.xml? Currently I set the path for namenode and datanode like... 
file:/home/user/hadoop_store/hdfs/namenode<\value> and similarly for 
datanode as well. In the log file for namenode, I see the line saying 
"Inconsistent state: storage directory doesn't exist or not accessible for the 
path /home/user/hadoop_store/hdfs/namenode.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747369#comment-16747369
 ] 

Hadoop QA commented on HDFS-14216:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m 10s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
49s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
49s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 
0 new + 134 unchanged - 1 fixed = 134 total (was 135) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m  2s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 20s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
32s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}154m 13s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.server.datanode.TestDataNodeErasureCodingMetrics |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14216 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12955537/HDFS-14216_3.patch |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux dd0d7476517e 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 824dfa3 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/26013/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/26013/testReport/ |
| Max. process+thread count | 3022 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output |

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Description: 
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is {color:#d04437}just  wiped before line280{color}, 
or{color:#33} {color}{color:#ff}we{color}{color:#ff} give the wrong 
DN name{color}*,*then  bm.getDatanodeManager().getDatanodeByHost(host) will 
return null, *_excludes_* *containes null*. while *_excludes_* are used later, 
NPE happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
{code}
 

  was:
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wiped before line280, 
or{color:#33}{color:#ff} {color}{color:#ff}we{color}{color:#ff} 
give the wrong DN name{color}{color}*,*then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
*containes null*. while *_excludes_* are used later, NPE happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



[ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747323#comment-16747323
 ] 

lujie commented on HDFS-14216:
--

HI:[~ayushtkn]

Reattach the patch which include a new UT. The UT just simulate what happens in 
real workload. Local test works well, could you please review it?

Thanks!

> NullPointerException happens in NamenodeWebHdfs
> ---
>
> Key: HDFS-14216
> URL: https://issues.apache.org/jira/browse/HDFS-14216
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: lujie
>Priority: Critical
> Attachments: HDFS-14216_1.patch, HDFS-14216_2.patch, 
> HDFS-14216_3.patch, hadoop-hires-namenode-hadoop11.log
>
>
>  workload
> {code:java}
> curl -i -X PUT -T $HOMEPARH/test.txt 
> "http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
> {code}
> the method
> {code:java}
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
>  excludeDatanodes){
>     HashSet excludes = new HashSet();
> if (excludeDatanodes != null) {
>for (String host : StringUtils
>  .getTrimmedStringCollection(excludeDatanodes)) {
>  int idx = host.indexOf(":");
>if (idx != -1) { 
> excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
>host.substring(0, idx), Integer.parseInt(host.substring(idx + 
> 1;
>} else {
>   
> excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
>}
>   }
> }
> }
> {code}
> when datanode(e.g.hadoop2) is just  wiped before line280, 
> or{color:#33}{color:#ff} 
> {color}{color:#ff}we{color}{color:#ff} give the wrong DN 
> name{color}{color}*,*then  bm.getDatanodeManager().getDatanodeByHost(host) 
> will return null, *_excludes_* *containes null*. while *_excludes_* are used 
> later, NPE happens:
> {code:java}
> java.lang.NullPointerException
> at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
> at 
> org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Attachment: HDFS-14216_3.patch

> NullPointerException happens in NamenodeWebHdfs
> ---
>
> Key: HDFS-14216
> URL: https://issues.apache.org/jira/browse/HDFS-14216
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: lujie
>Priority: Critical
> Attachments: HDFS-14216_1.patch, HDFS-14216_2.patch, 
> HDFS-14216_3.patch, hadoop-hires-namenode-hadoop11.log
>
>
>  workload
> {code:java}
> curl -i -X PUT -T $HOMEPARH/test.txt 
> "http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
> {code}
> the method
> {code:java}
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
>  excludeDatanodes){
>     HashSet excludes = new HashSet();
> if (excludeDatanodes != null) {
>for (String host : StringUtils
>  .getTrimmedStringCollection(excludeDatanodes)) {
>  int idx = host.indexOf(":");
>if (idx != -1) { 
> excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
>host.substring(0, idx), Integer.parseInt(host.substring(idx + 
> 1;
>} else {
>   
> excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
>}
>   }
> }
> }
> {code}
> when datanode(e.g.hadoop2) is just  wiped before line280, 
> or{color:#33}{color:#ff} 
> {color}{color:#ff}we{color}{color:#ff} give the wrong DN 
> name{color}{color}*,*then  bm.getDatanodeManager().getDatanodeByHost(host) 
> will return null, *_excludes_* *containes null*. while *_excludes_* are used 
> later, NPE happens:
> {code:java}
> java.lang.NullPointerException
> at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
> at 
> org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-13794) [PROVIDED Phase 2] Teach BlockAliasMap.Writer `remove` method.

2019-01-19 Thread Virajith Jalaparti (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Virajith Jalaparti updated HDFS-13794:
--
Status: Open  (was: Patch Available)

> [PROVIDED Phase 2] Teach BlockAliasMap.Writer `remove` method.
> --
>
> Key: HDFS-13794
> URL: https://issues.apache.org/jira/browse/HDFS-13794
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Ewan Higgs
>Assignee: Ewan Higgs
>Priority: Major
> Attachments: HDFS-13794-HDFS-12090.001.patch, 
> HDFS-13794-HDFS-12090.002.patch, HDFS-13794-HDFS-12090.003.patch, 
> HDFS-13794-HDFS-12090.004.patch, HDFS-13794-HDFS-12090.005.patch
>
>
> When updating the BlockAliasMap we may need to deal with deleted blocks. 
> Otherwise the BlockAliasMap will grow indefinitely(!).
> Therefore, the BlockAliasMap.Writer needs a method for removing blocks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-13794) [PROVIDED Phase 2] Teach BlockAliasMap.Writer `remove` method.

2019-01-19 Thread Virajith Jalaparti (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-13794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Virajith Jalaparti updated HDFS-13794:
--
Status: Patch Available  (was: Open)

> [PROVIDED Phase 2] Teach BlockAliasMap.Writer `remove` method.
> --
>
> Key: HDFS-13794
> URL: https://issues.apache.org/jira/browse/HDFS-13794
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Ewan Higgs
>Assignee: Ewan Higgs
>Priority: Major
> Attachments: HDFS-13794-HDFS-12090.001.patch, 
> HDFS-13794-HDFS-12090.002.patch, HDFS-13794-HDFS-12090.003.patch, 
> HDFS-13794-HDFS-12090.004.patch, HDFS-13794-HDFS-12090.005.patch
>
>
> When updating the BlockAliasMap we may need to deal with deleted blocks. 
> Otherwise the BlockAliasMap will grow indefinitely(!).
> Therefore, the BlockAliasMap.Writer needs a method for removing blocks.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-14156) RBF: rollEdit() command fails with Router

2019-01-19 Thread JIRA



[ 
https://issues.apache.org/jira/browse/HDFS-14156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747296#comment-16747296
 ] 

Íñigo Goiri commented on HDFS-14156:


Thanks [~shubham.dewan] for the patch and [~ayushtkn] and [~surendrasingh] for 
the reviews.
Committed to HDFS-13891.

> RBF: rollEdit() command fails with Router
> -
>
> Key: HDFS-14156
> URL: https://issues.apache.org/jira/browse/HDFS-14156
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Shubham Dewan
>Priority: Major
>  Labels: RBF
> Fix For: HDFS-13891
>
> Attachments: HDFS-14156-HDFS-13891.006.patch, 
> HDFS-14156-HDFS-13891.007.patch, HDFS-14156.001.patch, HDFS-14156.002.patch, 
> HDFS-14156.003.patch, HDFS-14156.004.patch, HDFS-14156.005.patch
>
>
> {noformat}
> bin> ./hdfs dfsadmin -rollEdits
> rollEdits: Cannot cast java.lang.Long to long
> bin>
> {noformat}
> Trace :-
> {noformat}
> org.apache.hadoop.ipc.RemoteException(java.lang.ClassCastException): Cannot 
> cast java.lang.Long to long
> at java.lang.Class.cast(Class.java:3369)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:1085)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:982)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterClientProtocol.rollEdits(RouterClientProtocol.java:900)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.rollEdits(RouterRpcServer.java:862)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.rollEdits(ClientNamenodeProtocolServerSideTranslatorPB.java:899)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1520)
> at org.apache.hadoop.ipc.Client.call(Client.java:1466)
> at org.apache.hadoop.ipc.Client.call(Client.java:1376)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.rollEdits(Unknown Source)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.rollEdits(ClientNamenodeProtocolTranslatorPB.java:804)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy12.rollEdits(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.rollEdits(DFSClient.java:2350)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.rollEdits(DistributedFileSystem.java:1550)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.rollEdits(DFSAdmin.java:850)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.run(DFSAdmin.java:2353)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.main(DFSAdmin.java:2568)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HDFS-14156) RBF: rollEdit() command fail with router

2019-01-19 Thread JIRA



 [ 
https://issues.apache.org/jira/browse/HDFS-14156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Íñigo Goiri updated HDFS-14156:
---
Summary: RBF: rollEdit() command fail with router  (was: RBF: RollEdit 
command fail with router)

> RBF: rollEdit() command fail with router
> 
>
> Key: HDFS-14156
> URL: https://issues.apache.org/jira/browse/HDFS-14156
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Shubham Dewan
>Priority: Major
>  Labels: RBF
> Attachments: HDFS-14156-HDFS-13891.006.patch, 
> HDFS-14156-HDFS-13891.007.patch, HDFS-14156.001.patch, HDFS-14156.002.patch, 
> HDFS-14156.003.patch, HDFS-14156.004.patch, HDFS-14156.005.patch
>
>
> {noformat}
> bin> ./hdfs dfsadmin -rollEdits
> rollEdits: Cannot cast java.lang.Long to long
> bin>
> {noformat}
> Trace :-
> {noformat}
> org.apache.hadoop.ipc.RemoteException(java.lang.ClassCastException): Cannot 
> cast java.lang.Long to long
> at java.lang.Class.cast(Class.java:3369)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:1085)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:982)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterClientProtocol.rollEdits(RouterClientProtocol.java:900)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.rollEdits(RouterRpcServer.java:862)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.rollEdits(ClientNamenodeProtocolServerSideTranslatorPB.java:899)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1520)
> at org.apache.hadoop.ipc.Client.call(Client.java:1466)
> at org.apache.hadoop.ipc.Client.call(Client.java:1376)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.rollEdits(Unknown Source)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.rollEdits(ClientNamenodeProtocolTranslatorPB.java:804)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy12.rollEdits(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.rollEdits(DFSClient.java:2350)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.rollEdits(DistributedFileSystem.java:1550)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.rollEdits(DFSAdmin.java:850)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.run(DFSAdmin.java:2353)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.main(DFSAdmin.java:2568)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For

[jira] [Updated] (HDFS-14156) RBF: rollEdit() command fails with Router

2019-01-19 Thread JIRA



 [ 
https://issues.apache.org/jira/browse/HDFS-14156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Íñigo Goiri updated HDFS-14156:
---
Summary: RBF: rollEdit() command fails with Router  (was: RBF: rollEdit() 
command fail with router)

> RBF: rollEdit() command fails with Router
> -
>
> Key: HDFS-14156
> URL: https://issues.apache.org/jira/browse/HDFS-14156
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Shubham Dewan
>Priority: Major
>  Labels: RBF
> Attachments: HDFS-14156-HDFS-13891.006.patch, 
> HDFS-14156-HDFS-13891.007.patch, HDFS-14156.001.patch, HDFS-14156.002.patch, 
> HDFS-14156.003.patch, HDFS-14156.004.patch, HDFS-14156.005.patch
>
>
> {noformat}
> bin> ./hdfs dfsadmin -rollEdits
> rollEdits: Cannot cast java.lang.Long to long
> bin>
> {noformat}
> Trace :-
> {noformat}
> org.apache.hadoop.ipc.RemoteException(java.lang.ClassCastException): Cannot 
> cast java.lang.Long to long
> at java.lang.Class.cast(Class.java:3369)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:1085)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:982)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterClientProtocol.rollEdits(RouterClientProtocol.java:900)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.rollEdits(RouterRpcServer.java:862)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.rollEdits(ClientNamenodeProtocolServerSideTranslatorPB.java:899)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1520)
> at org.apache.hadoop.ipc.Client.call(Client.java:1466)
> at org.apache.hadoop.ipc.Client.call(Client.java:1376)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.rollEdits(Unknown Source)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.rollEdits(ClientNamenodeProtocolTranslatorPB.java:804)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy12.rollEdits(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.rollEdits(DFSClient.java:2350)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.rollEdits(DistributedFileSystem.java:1550)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.rollEdits(DFSAdmin.java:850)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.run(DFSAdmin.java:2353)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.main(DFSAdmin.java:2568)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org

[jira] [Commented] (HDFS-14179) BlockReaderRemote#readNextPacket() should log the waiting time for packet read.

2019-01-19 Thread JIRA



[ 
https://issues.apache.org/jira/browse/HDFS-14179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747297#comment-16747297
 ] 

Íñigo Goiri commented on HDFS-14179:


The main overhead would be the two new {{System.currentTimeMillis()}} calls, 
right?
The log messages are guarded so it should be fairly fast.
[~shubham.dewan] can you take some quick performance numbers for the approach?


> BlockReaderRemote#readNextPacket() should log the waiting time for packet 
> read.
> ---
>
> Key: HDFS-14179
> URL: https://issues.apache.org/jira/browse/HDFS-14179
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs-client
>Affects Versions: 3.0.0
>Reporter: Surendra Singh Lilhore
>Assignee: Shubham Dewan
>Priority: Major
>  Labels: newbie
> Attachments: HDFS-14179.001.patch, HDFS-14179.002.patch, 
> HDFS-14179.003.patch
>
>
> Sometime read is reported very slow due to disk or some other reason. 
> {{BlockReaderRemote#readNextPacket()}} should print the datanode IP and 
> waiting time in trace log.
> {code:java}
> //Read packet headers.
> packetReceiver.receiveNextPacket(in);
> PacketHeader curHeader = packetReceiver.getHeader();
> curDataSlice = packetReceiver.getDataSlice();
> assert curDataSlice.capacity() == curHeader.getDataLen();
> LOG.trace("DFSClient readNextPacket got header {}", curHeader);{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-14156) RBF: rollEdit() command fails with Router

2019-01-19 Thread JIRA



 [ 
https://issues.apache.org/jira/browse/HDFS-14156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Íñigo Goiri updated HDFS-14156:
---
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: HDFS-13891
   Status: Resolved  (was: Patch Available)

> RBF: rollEdit() command fails with Router
> -
>
> Key: HDFS-14156
> URL: https://issues.apache.org/jira/browse/HDFS-14156
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Shubham Dewan
>Priority: Major
>  Labels: RBF
> Fix For: HDFS-13891
>
> Attachments: HDFS-14156-HDFS-13891.006.patch, 
> HDFS-14156-HDFS-13891.007.patch, HDFS-14156.001.patch, HDFS-14156.002.patch, 
> HDFS-14156.003.patch, HDFS-14156.004.patch, HDFS-14156.005.patch
>
>
> {noformat}
> bin> ./hdfs dfsadmin -rollEdits
> rollEdits: Cannot cast java.lang.Long to long
> bin>
> {noformat}
> Trace :-
> {noformat}
> org.apache.hadoop.ipc.RemoteException(java.lang.ClassCastException): Cannot 
> cast java.lang.Long to long
> at java.lang.Class.cast(Class.java:3369)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:1085)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:982)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterClientProtocol.rollEdits(RouterClientProtocol.java:900)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.rollEdits(RouterRpcServer.java:862)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.rollEdits(ClientNamenodeProtocolServerSideTranslatorPB.java:899)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1520)
> at org.apache.hadoop.ipc.Client.call(Client.java:1466)
> at org.apache.hadoop.ipc.Client.call(Client.java:1376)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.rollEdits(Unknown Source)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.rollEdits(ClientNamenodeProtocolTranslatorPB.java:804)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy12.rollEdits(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.rollEdits(DFSClient.java:2350)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.rollEdits(DistributedFileSystem.java:1550)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.rollEdits(DFSAdmin.java:850)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.run(DFSAdmin.java:2353)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.main(DFSAdmin.java:2568)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747219#comment-16747219
 ] 

Hadoop QA commented on HDFS-14216:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 1 new or modified test 
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m 
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m 28s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
51s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 
0 new + 134 unchanged - 1 fixed = 134 total (was 135) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m 14s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
46s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}118m 54s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
44s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 29s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.server.namenode.TestFSImage |
|   | hadoop.hdfs.server.balancer.TestBalancer |
|   | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14216 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12955525/HDFS-14216_2.patch |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 3446170baa71 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 824dfa3 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/26012/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/26012/testReport/ |
| Max. process+thread count | 3082 (vs. ulimit of 1) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs

[jira] [Comment Edited] (HDDS-761) Create S3 subcommand to run S3 related operations

2019-01-19 Thread Bharat Viswanadham (JIRA)



[ 
https://issues.apache.org/jira/browse/HDDS-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747216#comment-16747216
 ] 

Bharat Viswanadham edited comment on HDDS-761 at 1/19/19 5:37 PM:
--

Hi [~ste...@apache.org]
We have a bucket mapping command to provide s3bucket -> bucket/volume mapping. 
Want to move this command from ozone sh bucket path <> to ozone s3 
bucket path <>
And currently under this, we have getSecret to get acessKeyId and secretKey.

And remaining all operations like bucket info, keys, everything else we can get 
via normal ozone shell commands. 




was (Author: bharatviswa):
Hi [~ste...@apache.org]
We have a bucket mapping command to provide s3bucket -> bucket/volume mapping.
And currently under this, we have getSecret to get acessKeyId and secretKey.

And remaining all operations like bucket info, keys, everything else we can get 
via ozone shell commands. 



> Create S3 subcommand to run S3 related operations
> -
>
> Key: HDDS-761
> URL: https://issues.apache.org/jira/browse/HDDS-761
> Project: Hadoop Distributed Data Store
>  Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
>
> This Jira is added to create S3 subcommand, which will be used for all S3 
> related operations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDDS-761) Create S3 subcommand to run S3 related operations

2019-01-19 Thread Bharat Viswanadham (JIRA)



[ 
https://issues.apache.org/jira/browse/HDDS-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747216#comment-16747216
 ] 

Bharat Viswanadham commented on HDDS-761:
-

Hi [~ste...@apache.org]
We have a bucket mapping command to provide s3bucket -> bucket/volume mapping.
And currently under this, we have getSecret to get acessKeyId and secretKey.

And remaining all operations like bucket info, keys, everything else we can get 
via ozone shell commands. 



> Create S3 subcommand to run S3 related operations
> -
>
> Key: HDDS-761
> URL: https://issues.apache.org/jira/browse/HDDS-761
> Project: Hadoop Distributed Data Store
>  Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
>
> This Jira is added to create S3 subcommand, which will be used for all S3 
> related operations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDDS-761) Create S3 subcommand to run S3 related operations

2019-01-19 Thread Steve Loughran (JIRA)



[ 
https://issues.apache.org/jira/browse/HDDS-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747188#comment-16747188
 ] 

Steve Loughran commented on HDDS-761:
-

what kind of operations are you imagining? As we have a set of ops under the 
"hadoop s3guard" command such as bucket-info -and I'm looking at a get-metadata 
one to query metadata, primarily for diagnostics and testing

> Create S3 subcommand to run S3 related operations
> -
>
> Key: HDDS-761
> URL: https://issues.apache.org/jira/browse/HDDS-761
> Project: Hadoop Distributed Data Store
>  Issue Type: Sub-task
>Reporter: Bharat Viswanadham
>Assignee: Bharat Viswanadham
>Priority: Major
>
> This Jira is added to create S3 subcommand, which will be used for all S3 
> related operations.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread Ayush Saxena (JIRA)



[ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747141#comment-16747141
 ] 

Ayush Saxena commented on HDFS-14216:
-

Thanx [~xiaoheipangzi] for the patch.

The fix seems quite straightforward.

Can we extend a proper unit test rather than hampering the existing one,which 
can give a check to the NPE. :)



> NullPointerException happens in NamenodeWebHdfs
> ---
>
> Key: HDFS-14216
> URL: https://issues.apache.org/jira/browse/HDFS-14216
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: lujie
>Priority: Critical
> Attachments: HDFS-14216_1.patch, HDFS-14216_2.patch, 
> hadoop-hires-namenode-hadoop11.log
>
>
>  workload
> {code:java}
> curl -i -X PUT -T $HOMEPARH/test.txt 
> "http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
> {code}
> the method
> {code:java}
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
>  excludeDatanodes){
>     HashSet excludes = new HashSet();
> if (excludeDatanodes != null) {
>for (String host : StringUtils
>  .getTrimmedStringCollection(excludeDatanodes)) {
>  int idx = host.indexOf(":");
>if (idx != -1) { 
> excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
>host.substring(0, idx), Integer.parseInt(host.substring(idx + 
> 1;
>} else {
>   
> excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
>}
>   }
> }
> }
> {code}
> when datanode(e.g.hadoop2) is just  wiped before line280, 
> or{color:#33}{color:#ff} 
> {color}{color:#ff}we{color}{color:#ff} give the wrong DN 
> name{color}{color}*,*then  bm.getDatanodeManager().getDatanodeByHost(host) 
> will return null, *_excludes_* *containes null*. while *_excludes_* are used 
> later, NPE happens:
> {code:java}
> java.lang.NullPointerException
> at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
> at 
> org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Description: 
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wiped before line280, 
or{color:#33}{color:#ff} {color}{color:#ff}we{color}{color:#ff} 
give the wrong DN name{color}{color}*,*then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
*containes null*. while *_excludes_* are used later, NPE happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
{code}
 

  was:
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wiped before line280, or{color:#FF} 
*{color:#ff}we give the wrong DN name{color}*{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Description: 
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wiped before line280, or{color:#ff} we 
give the wrong DN name{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
{code}
 

  was:
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wipe before line280, or{color:#ff} we 
give the wrong DN name{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Description: 
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wiped before line280, or{color:#FF} 
*{color:#ff}we give the wrong DN name{color}*{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
{code}
 

  was:
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wiped before line280, or{color:#ff} we 
give the wrong DN name{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Attachment: HDFS-14216_2.patch

> NullPointerException happens in NamenodeWebHdfs
> ---
>
> Key: HDFS-14216
> URL: https://issues.apache.org/jira/browse/HDFS-14216
> Project: Hadoop HDFS
>  Issue Type: Bug
>Reporter: lujie
>Priority: Critical
> Attachments: HDFS-14216_1.patch, HDFS-14216_2.patch, 
> hadoop-hires-namenode-hadoop11.log
>
>
>  workload
> {code:java}
> curl -i -X PUT -T $HOMEPARH/test.txt 
> "http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
> {code}
> the method
> {code:java}
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
>  excludeDatanodes){
>     HashSet excludes = new HashSet();
> if (excludeDatanodes != null) {
>for (String host : StringUtils
>  .getTrimmedStringCollection(excludeDatanodes)) {
>  int idx = host.indexOf(":");
>if (idx != -1) { 
> excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
>host.substring(0, idx), Integer.parseInt(host.substring(idx + 
> 1;
>} else {
>   
> excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
>}
>   }
> }
> }
> {code}
> when datanode(e.g.hadoop2) is just  wipe before line280, or{color:#ff} we 
> give the wrong DN name{color},then  
> bm.getDatanodeManager().getDatanodeByHost(host) will return null, 
> *_excludes_* containes *{color:#ff}null{color}*. while *_excludes_* are 
> used later, NPE happens:
> {code:java}
> java.lang.NullPointerException
> at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
> at 
> org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
> at 
> org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
> at 
> org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
> at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
> {code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Description: 
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) is just  wipe before line280, or{color:#ff} we 
give the wrong DN name{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
{code}
 

  was:
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) just crashed before line280, or{color:#FF} we 
give the wrong DN name{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at

[jira] [Commented] (HDFS-14179) BlockReaderRemote#readNextPacket() should log the waiting time for packet read.

2019-01-19 Thread Ayush Saxena (JIRA)



[ 
https://issues.apache.org/jira/browse/HDFS-14179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747082#comment-16747082
 ] 

Ayush Saxena commented on HDFS-14179:
-

Thanx Everyone for the work here.

I am having a minor doubt on this, Whether this will not impact the performance 
of read process.This seems to be adding some extra processing overhead during 
the time of packet read for the LOG purpose. Though the overhead may seem quite 
minor but this is at packet level,at block level it shall magnify,at file level 
even more and so on.

If I am not missing on or getting into some wrong zone.This would effect the 
read performance and read being an elementary operation I am not sure whether 
we can afford it .Even can't say whether this log will bring that much value to 
us as compared to the cost it would take.

Pls correct me if I have misunderstood something on this context. :)

> BlockReaderRemote#readNextPacket() should log the waiting time for packet 
> read.
> ---
>
> Key: HDFS-14179
> URL: https://issues.apache.org/jira/browse/HDFS-14179
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs-client
>Affects Versions: 3.0.0
>Reporter: Surendra Singh Lilhore
>Assignee: Shubham Dewan
>Priority: Major
>  Labels: newbie
> Attachments: HDFS-14179.001.patch, HDFS-14179.002.patch, 
> HDFS-14179.003.patch
>
>
> Sometime read is reported very slow due to disk or some other reason. 
> {{BlockReaderRemote#readNextPacket()}} should print the datanode IP and 
> waiting time in trace log.
> {code:java}
> //Read packet headers.
> packetReceiver.receiveNextPacket(in);
> PacketHeader curHeader = packetReceiver.getHeader();
> curDataSlice = packetReceiver.getDataSlice();
> assert curDataSlice.capacity() == curHeader.getDataLen();
> LOG.trace("DFSClient readNextPacket got header {}", curHeader);{code}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-14156) RBF: RollEdit command fail with router

2019-01-19 Thread Ayush Saxena (JIRA)



[ 
https://issues.apache.org/jira/browse/HDFS-14156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16747081#comment-16747081
 ] 

Ayush Saxena commented on HDFS-14156:
-

No further additions from my Side... 

> RBF: RollEdit command fail with router
> --
>
> Key: HDFS-14156
> URL: https://issues.apache.org/jira/browse/HDFS-14156
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Affects Versions: 3.1.1
>Reporter: Harshakiran Reddy
>Assignee: Shubham Dewan
>Priority: Major
>  Labels: RBF
> Attachments: HDFS-14156-HDFS-13891.006.patch, 
> HDFS-14156-HDFS-13891.007.patch, HDFS-14156.001.patch, HDFS-14156.002.patch, 
> HDFS-14156.003.patch, HDFS-14156.004.patch, HDFS-14156.005.patch
>
>
> {noformat}
> bin> ./hdfs dfsadmin -rollEdits
> rollEdits: Cannot cast java.lang.Long to long
> bin>
> {noformat}
> Trace :-
> {noformat}
> org.apache.hadoop.ipc.RemoteException(java.lang.ClassCastException): Cannot 
> cast java.lang.Long to long
> at java.lang.Class.cast(Class.java:3369)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:1085)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcClient.invokeConcurrent(RouterRpcClient.java:982)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterClientProtocol.rollEdits(RouterClientProtocol.java:900)
> at 
> org.apache.hadoop.hdfs.server.federation.router.RouterRpcServer.rollEdits(RouterRpcServer.java:862)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.rollEdits(ClientNamenodeProtocolServerSideTranslatorPB.java:899)
> at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
> at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)
> at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1520)
> at org.apache.hadoop.ipc.Client.call(Client.java:1466)
> at org.apache.hadoop.ipc.Client.call(Client.java:1376)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:229)
> at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
> at com.sun.proxy.$Proxy11.rollEdits(Unknown Source)
> at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.rollEdits(ClientNamenodeProtocolTranslatorPB.java:804)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:422)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:165)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:157)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
> at 
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:359)
> at com.sun.proxy.$Proxy12.rollEdits(Unknown Source)
> at org.apache.hadoop.hdfs.DFSClient.rollEdits(DFSClient.java:2350)
> at 
> org.apache.hadoop.hdfs.DistributedFileSystem.rollEdits(DistributedFileSystem.java:1550)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.rollEdits(DFSAdmin.java:850)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.run(DFSAdmin.java:2353)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:90)
> at org.apache.hadoop.hdfs.tools.DFSAdmin.main(DFSAdmin.java:2568)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands,

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread Hadoop QA (JIRA)



[ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746988#comment-16746988
 ] 

Hadoop QA commented on HDFS-14216:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 
 5s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
14m  8s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
53s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
48s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 
0 new + 123 unchanged - 1 fixed = 123 total (was 124) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
13m  6s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 95m 14s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
32s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}155m 10s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | HDFS-14216 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12955501/HDFS-14216_1.patch |
| Optional Tests |  dupname  asflicense  compile  javac  javadoc  mvninstall  
mvnsite  unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 646344de5d39 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14 
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 824dfa3 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/26011/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/26011/testReport/ |
| Max. process+thread count | 3422 (vs. ulimit of 1) |
|

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Description: 
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) just crashed before line280, or{color:#FF} we 
give the wrong DN name{color},then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
{code}
 

  was:
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) just crashed before line280, or we give the wrong DN 
name,then  bm.getDatanodeManager().getDatanodeByHost(host) will return null, 
*_excludes_* containes *{color:#ff}null{color}*. while *_excludes_* are 
used later, NPE happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-01-19 Thread lujie (JIRA)



 [ 
https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

lujie updated HDFS-14216:
-
Description: 
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) just crashed before line280, or we give the wrong DN 
name,then  bm.getDatanodeManager().getDatanodeByHost(host) will return null, 
*_excludes_* containes *{color:#ff}null{color}*. while *_excludes_* are 
used later, NPE happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)
{code}
 

  was:
 workload
{code:java}
curl -i -X PUT -T $HOMEPARH/test.txt 
"http://hadoop1:9870/webhdfs/v1/input?op=CREATE=hadoop2;
{code}
the method
{code:java}
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String
 excludeDatanodes){

    HashSet excludes = new HashSet();
if (excludeDatanodes != null) {
   for (String host : StringUtils
 .getTrimmedStringCollection(excludeDatanodes)) {
 int idx = host.indexOf(":");
   if (idx != -1) { 
excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
   host.substring(0, idx), Integer.parseInt(host.substring(idx + 1;
   } else {
  
excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
   }
  }
}

}
{code}
when datanode(e.g.hadoop2) just crashed before line280, then  
bm.getDatanodeManager().getDatanodeByHost(host) will return null, *_excludes_* 
containes *{color:#ff}null{color}*. while *_excludes_* are used later, NPE 
happens:
{code:java}
java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at 
org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at 
org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)

[jira] [Updated] (HDFS-14217) namenode and datanode are not starting

[jira] [Created] (HDFS-14217) namenode and datanode are not starting

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-13794) [PROVIDED Phase 2] Teach BlockAliasMap.Writer `remove` method.

[jira] [Updated] (HDFS-13794) [PROVIDED Phase 2] Teach BlockAliasMap.Writer `remove` method.

[jira] [Commented] (HDFS-14156) RBF: rollEdit() command fails with Router

[jira] [Updated] (HDFS-14156) RBF: rollEdit() command fail with router

[jira] [Updated] (HDFS-14156) RBF: rollEdit() command fails with Router

[jira] [Commented] (HDFS-14179) BlockReaderRemote#readNextPacket() should log the waiting time for packet read.

[jira] [Updated] (HDFS-14156) RBF: rollEdit() command fails with Router

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Comment Edited] (HDDS-761) Create S3 subcommand to run S3 related operations

[jira] [Commented] (HDDS-761) Create S3 subcommand to run S3 related operations

[jira] [Commented] (HDDS-761) Create S3 subcommand to run S3 related operations

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Commented] (HDFS-14179) BlockReaderRemote#readNextPacket() should log the waiting time for packet read.

[jira] [Commented] (HDFS-14156) RBF: RollEdit command fail with router

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

28 matches

Site Navigation

Mail list logo

Footer information