[jira] [Commented] (HADOOP-9585) unit test failure :org.apache.hadoop.fs.TestFsShellReturnCode.testChgrp

2013-06-20 Thread Leo Leung (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-9585?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13689602#comment-13689602
 ] 

Leo Leung commented on HADOOP-9585:
---

Current test case assumes jenkins (or the test runner) is part of the admin 
group.
Suggest to change test case to add 
1) positive test scenario that includes the users' group
2) and negative scenario that includes none-existing-group. (invalid group)

something like:
+String[] groups = UserGroupInformation.getCurrentUser().getGroupNames();
-String argv[] = { -chgrp, admin, f1 };
-verify(fs, -chgrp, argv, 1, fsShell, 0);
+for (String member : groups)
+{
+  String argv[] = { -chgrp, member, f1 };
+  verify(fs, -chgrp, argv, 1, fsShell, 0);
+}
-// Test 2: exit code for chgrp on non existing path is 1
+// Test 2: exit code for non-existing group on existing file
+String argv1[] = { -chgrp, groupdoesnotexist, f1 };
+verify(fs, -chgrp, argv1, 1, fsShell, 1);




 unit test failure :org.apache.hadoop.fs.TestFsShellReturnCode.testChgrp
 ---

 Key: HADOOP-9585
 URL: https://issues.apache.org/jira/browse/HADOOP-9585
 Project: Hadoop Common
  Issue Type: Bug
  Components: test
Affects Versions: 1.3.0
 Environment: 
 https://builds.apache.org/job/Hadoop-branch1/lastCompletedBuild/testReport/org.apache.hadoop.fs/TestFsShellReturnCode/testChgrp/
Reporter: Giridharan Kesavan

 Standard Error
 chmod: could not get status for 
 '/home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChmod/fileDoesNotExist':
  File 
 /home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChmod/fileDoesNotExist
  does not exist.
 chmod: could not get status for 
 '/home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChmod/nonExistingfiles*'
 chown: could not get status for 
 '/home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChown/fileDoesNotExist':
  File 
 /home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChown/fileDoesNotExist
  does not exist.
 chown: could not get status for 
 '/home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChown/nonExistingfiles*'
 chgrp: failed on 
 'file:/home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChgrp/fileExists':
  chgrp: changing group of 
 `/home/jenkins/jenkins-slave/workspace/Hadoop-branch1/trunk/build/test/data/testChgrp/fileExists':
  Operation not permitted

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HADOOP-9586) unit test failure: org.apache.hadoop.hdfs.TestFileCreation.testFileCreationSetLocalInterface

2013-06-18 Thread Leo Leung (JIRA) 

[ 
https://issues.apache.org/jira/browse/HADOOP-9586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687404#comment-13687404
 ] 

Leo Leung commented on HADOOP-9586:
---

Could be related to DFSClient::getRandomLocalInterfaceAddr() picking up an IPv6 
IP for interface lo on a dual stack system. Changes seems to be from 
[HDFS-3148]

h3. The following fails:
2013-06-18 16:34:30,141 DEBUG hdfs.DFSClient (DFSClient.java:init(372)) - 
Using local interfaces [lo] with addresses 
{color:red}[/0:0:0:0:0:0:0:1%1:0,/127.0.0.1:0]{color}
 [...]
2013-06-18 16:34:30,289 DEBUG hdfs.DFSClient 
(DFSClient.java:createBlockOutputStream(3852)) - Connecting to 127.0.0.1:51117
2013-06-18 16:34:30,289 DEBUG hdfs.DFSClient 
(DFSClient.java:getRandomLocalInterfaceAddr(1204)) - Using local interface 
{color:red}*/0:0:0:0:0:0:0:1%1:0*{color}
2013-06-18 16:34:30,289 INFO  hdfs.DFSClient 
(DFSClient.java:createBlockOutputStream(3904)) - Exception in 
createBlockOutputStream 127.0.0.1:51117 java.net.SocketException: Network is 
unreachable
2013-06-18 16:34:30,290 INFO  hdfs.DFSClient 
(DFSClient.java:nextBlockOutputStream(3810)) - Abandoning 
blk_7035064663379527570_1001
2013-06-18 16:34:30,292 INFO  hdfs.DFSClient 
(DFSClient.java:nextBlockOutputStream(3814)) - Excluding datanode 
127.0.0.1:51117
2013-06-18 16:34:30,294 WARN  namenode.FSNamesystem 
(BlockPlacementPolicyDefault.java:chooseTarget(209)) - Not able to place enough 
replicas, still in need of 1 to reach 1
Not able to place enough replicas
2013-06-18 16:34:30,294 ERROR security.UserGroupInformation 
(UserGroupInformation.java:doAs(1235)) - PriviledgedActionException as:ddn 
cause:java.io.IOException: File /user/ddn/filestatus.dat could only be 
replicated to 0 nodes, instead of 1
2013-06-18 16:34:30,294 INFO  ipc.Server (Server.java:run(1463)) - IPC Server 
handler 8 on 41922, call addBlock(/user/ddn/filestatus.dat, 
DFSClient_NONMAPREDUCE_-97075681_1, 
[Lorg.apache.hadoop.hdfs.protocol.DatanodeInfo;@b04bce3) from 127.0.0.1:48545: 
error: java.io.IOException: File /user/ddn/filestatus.dat could only be 
replicated to 0 nodes, instead of 1
java.io.IOException: File /user/ddn/filestatus.dat could only be replicated to 
0 nodes, instead of 1
at 
org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1983)

h3. The following works: 
[root@host] echo 1  /proc/sys/net/ipv6/conf/all/disable_ipv6 

2013-06-18 16:52:41,890 DEBUG hdfs.DFSClient (DFSClient.java:init(372)) - 
Using local interfaces [lo] with addresses [/127.0.0.1:0]
2013-06-18 16:52:42,029 DEBUG hdfs.DFSClient 
(DFSClient.java:createBlockOutputStream(3852)) - Connecting to 127.0.0.1:45078
2013-06-18 16:52:42,029 DEBUG hdfs.DFSClient 
(DFSClient.java:getRandomLocalInterfaceAddr(1204)) - Using local interface 
/127.0.0.1:0
2013-06-18 16:52:42,029 DEBUG hdfs.DFSClient 
(DFSClient.java:createBlockOutputStream(3856)) - Send buf size 131072
2013-06-18 16:52:42,030 INFO  datanode.DataNode 
(DataXceiver.java:writeBlock(249)) - Receiving blk_4193380550170530321_1001 
src: /127.0.0.1:60010 dest: /127.0.0.1:45078
2013-06-18 16:52:42,032 DEBUG hdfs.DFSClient (DFSClient.java:run(3277)) - 
DataStreamer block blk_4193380550170530321_1001 wrote packet seqno:0 size:8281 
offsetInBlock:0 lastPacketInBlock:true


h3. Summary/Proposal:
Need a fix for DFSClient to understand and support (or not) IPv6



 unit test failure: 
 org.apache.hadoop.hdfs.TestFileCreation.testFileCreationSetLocalInterface
 

 Key: HADOOP-9586
 URL: https://issues.apache.org/jira/browse/HADOOP-9586
 Project: Hadoop Common
  Issue Type: Bug
  Components: test
Affects Versions: 1.3.0
Reporter: Giridharan Kesavan

 https://builds.apache.org/job/Hadoop-branch1/lastCompletedBuild/testReport/org.apache.hadoop.hdfs/TestFileCreation/testFileCreationSetLocalInterface/
 org.apache.hadoop.ipc.RemoteException: java.io.IOException: File 
 /user/jenkins/filestatus.dat could only be replicated to 0 nodes, instead of 1
   at 
 org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920)
   at 
 org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783)
   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)
   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)
   at java.security.AccessController.doPrivileged(Native Method)
   at javax.security.auth.Subject.doAs(Subject.java:396)
   at 
 org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
   at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)
   at org.apache.hadoop.ipc.Client.call(Client.java:1107)
   at