Hi This Could be due to the Following reason 1) The *NameNode <http://wiki.apache.org/hadoop/NameNode>* does not have any available DataNodes 2) Namenode not able to start properly 3) other wise some IP Issue . Note:- Pleaes mention localhost instead of 127.0.0.1 (If it is in local)
Follow URL: http://wiki.apache.org/hadoop/FAQ#What_does_.22file_could_only_be_replicated_to_0_nodes.2C_instead_of_1.22_mean.3F Thanks samir On Sat, May 19, 2012 at 11:30 PM, samir das mohapatra < samir.help...@gmail.com> wrote: > Hi > This Could be due to the Following reason > > 1) The *NameNode <http://wiki.apache.org/hadoop/NameNode>* does not have > any available DataNodes > 2) Namenode not able to start properly > 3) other wise some IP Issue . > Note:- Pleaes mention localhost instead of 127.0.0.1 (If it is in > local) > > Follow URL: > > http://wiki.apache.org/hadoop/FAQ#What_does_.22file_could_only_be_replicated_to_0_nodes.2C_instead_of_1.22_mean.3F > > > Thanks > samir > > > > On Sat, May 19, 2012 at 8:59 PM, Todd McFarland <toddmcf2...@gmail.com>wrote: > >> Hi folks, >> >> (Resending to this group, sent to common-dev before, pretty sure that's >> for >> Hadoop internal development - sorry for that..) >> >> I'm pretty stuck here. I've been researching for hours and I haven't made >> any forward progress on this one. >> >> I have a vmWare installation of Cloudera Hadoop 0.20. The following >> commands to create a directory and copy a file from the shared folder >> *work >> fine*, so I'm confident everything is setup correctly: >> >> [cloudera@localhost bin]$ hadoop fs -mkdir /user/cloudera/testdir >> [cloudera@localhost bin]$ hadoop fs -put >> /mnt/hgfs/shared_folder/file1.txt >> /user/cloudera/testdir/file1.txt >> >> The file shows up fine in the HDFS doing it this way on the Linux VM. >> >> *However*, when I try doing the equivalent operation in Java everything >> works great until I try to close() FSDataOutputStream. >> I'm left with the new directory and a zero byte size file. One suspicious >> thing is that the user is "admin" instead of "cloudera" which I haven't >> figured out why. Here is the error: >> >> 12/05/19 09:45:46 INFO hdfs.DFSClient: Exception in >> createBlockOutputStream >> 127.0.0.1:50010 java.net.ConnectException: Connection refused: no further >> information >> 12/05/19 09:45:46 INFO hdfs.DFSClient: Abandoning block >> blk_1931357292676354131_1068 >> 12/05/19 09:45:46 INFO hdfs.DFSClient: Excluding datanode 127.0.0.1:50010 >> 12/05/19 09:45:46 WARN hdfs.DFSClient: DataStreamer Exception: >> org.apache.hadoop.ipc.RemoteException: java.io.IOException: File >> /user/admin/testdir/file1.txt could only be replicated to 0 nodes, instead >> of 1 >> at >> >> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1533) >> at >> >> org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:667) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> >> There are certainly lots of search references to "*could only be >> replicated >> to 0 nodes, instead of 1*" but chasing down those suggestions hasn't >> helped. >> I have run *jps* and* netstat* and that looks good. All services are >> running, all port seem to be good. The *health check* looks good, plenty >> of disk space, no failed nodes... >> >> Here is the java (it fails when it hits "fs.close()": >> >> import java.io.BufferedReader; >> import java.io.FileInputStream; >> import java.io.FileReader; >> import java.io.IOException; >> import org.apache.hadoop.conf.Configuration; >> import org.apache.hadoop.fs.FSDataInputStream; >> import org.apache.hadoop.fs.FSDataOutputStream; >> import org.apache.hadoop.fs.FileSystem; >> import org.apache.hadoop.fs.Path; >> >> public class TestFileTrans { >> >> public static void main(String[] args) { >> >> Configuration config = new Configuration(); >> >> config.addResource(new >> Path("c:/_bigdata/client_libs/core-site.xml")); >> config.addResource(new >> Path("c:/_bigdata/client_libs/hdfs-site.xml")); >> >> System.out.println("hadoop.tmp.dir: " + >> config.get("hadoop.tmp.dir")); >> try{ >> FileSystem dfs = FileSystem.get(config); >> >> // this will default to admin unless the workingDirectory is >> explicitly set.. >> >> System.out.println("HDFS Working Directory: " + >> dfs.getWorkingDirectory().toString()); >> >> String dirName = "testdir"; >> Path src = new Path(dfs.getWorkingDirectory()+"/"+dirName); >> dfs.mkdirs(src); >> >> System.out.println("HDFS Directory created: " + >> dfs.getWorkingDirectory().toString()); >> >> loadFile(dfs, src); >> >> }catch(IOException e){ >> System.out.println("Error" + e.getMessage()); >> } >> >> } >> >> private static void loadFile(FileSystem dfs, Path src) throws >> IOException{ >> >> FileInputStream fis = new >> FileInputStream("c:/_bigdata/shared_folder/file1.txt"); >> >> int len = fis.available(); >> >> byte[] btr = new byte[len]; >> >> fis.read(btr); >> >> FSDataOutputStream fs = dfs.create(new Path(src.toString() >> +"/file1.txt")); >> >> fs.write(btr); >> >> fs.flush(); >> fs.close(); >> >> } >> } >> >> Any help would be greatly appreciated! >> > >
