Ravi, Thanks for your feedback. Currently, I changed YAML to bind a host port for HDFS, and use Mesos as resource management to avoid port conflict. It works for now, I’ll continue to investigate this issue and keep you posted.
Thanks Klaus On Jun 7, 2016, at 05:10, Ravi Prakash <[email protected]<mailto:[email protected]>> wrote: Klaus! Good luck with your attempt to run HDFS inside Kubernetes! Please keep us posted. For creating a new file, a DFSClient : 1. First calls addBlock on the NameNode. https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRpcServer.java#L842 . This returns a list of "LocatedBlocks" . (This is essentially a list of datanode storages to which the client should write) 2. The DFSClient then creates a pipeline to which it streams data consisting of the (usually 3) datanodes. In your case, when you go to the Namenode Web UI (http://<namenode>:50070/dfshealth.html#tab-datanode) what is the Datanode's ID? You should debug the client and namenode to see what is the list of LocatedBlocks returned by the addBlock call. HTH Ravi On Sat, Jun 4, 2016 at 7:22 AM, Klaus Ma <[email protected]<mailto:[email protected]>> wrote: Hi team, I'm working to run HDFS in kubernetes; all configuration is ready: kube-dns, hdfs-site.xml and ssh. But when I create files in HDFS I got the following exception. In exception, "10.0.1.126:50010<http://10.0.1.126:50010/>" is the host's ip & port instead of container; is there any configuration to ask DFSClient to use container's IP instead of host IP? 16/06/04 14:06:23 INFO hdfs.DFSClient: Exception in createBlockOutputStream java.net.ConnectException: Connection refused at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744) at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531) at org.apache.hadoop.hdfs.DFSOutputStream.createSocketForPipeline(DFSOutputStream.java:1537) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.createBlockOutputStream(DFSOutputStream.java:1313) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1266) at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:449) 16/06/04 14:06:23 INFO hdfs.DFSClient: Abandoning BP-223491250-172.1.78.2-1465048638628:blk_1073741825_1001 16/06/04 14:06:23 INFO hdfs.DFSClient: Excluding datanode DatanodeInfoWithStorage[10.0.1.126:50010<http://10.0.1.126:50010/>,DS-a2c2d3db-790c-4b76-81f6-856c809b01e2,DISK] 16/06/04 14:06:23 WARN hdfs.DFSClient: DataStreamer Exception org.apache.hadoop.ipc.RemoteException(java.io.IOException): File /user/root/QuasiMonteCarlo_1465049182505_2071986941/in/part0 could only be replicated to 0 nodes instead of minReplication (=1). There are 1 datanode(s) running and 1 node(s) are excluded in this operation. at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget4NewBlock(BlockManager.java:1547) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3107) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3031) at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:724) at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492) at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) ---- Da (Klaus), Ma (马达) | PMP | Advisory Software Engineer Platform OpenSource Technology, STG, IBM GCG +86-10-8245 4084<tel:%2B86-10-8245%204084> | [email protected]<mailto:[email protected]> | http://k82.me<http://k82.me/>
