Hi, I wrote a short Program on Hadoop Reading and Writing. I want to read a local file and write it to the hadoop. The code is below:
public class HadoopDFSFileReadWrite { public static void main(String[] argv) throws IOException { Configuration conf = new Configuration(); FileSystem fs = FileSystem.get(conf); FileSystem dfs = null; try { dfs = FileSystem.get(new URI("hdfs://root:[EMAIL PROTECTED]:50010"), conf); //????? can not connect to datanode Exception Below: /* Exception in thread "main" java.net.SocketTimeoutException: timed out waiting for rpc response at org.apache.hadoop.ipc.Client.call(Client.java:484) at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:184) at org.apache.hadoop.dfs.$Proxy0.getProtocolVersion(Unknown Source) at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:269) at org.apache.hadoop.dfs.DFSClient.createNamenode(DFSClient.java:147) at org.apache.hadoop.dfs.DFSClient.<init>(DFSClient.java:161) at org.apache.hadoop.dfs.DistributedFileSystem.initialize( DistributedFileSystem.java:65) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:159) at readWrite.HadoopDFSFileReadWrite.main(HadoopDFSFileReadWrite.java:50) */ } catch (URISyntaxException e1) { // TODO Auto-generated catch block e1.printStackTrace(); } if (argv.length != 2) usage(); // Hadoop DFS deals with Path Path inFile = new Path(argv[0]); Path outFile = new Path(argv[1]); // Check if input/output are valid if (!fs.exists(inFile)) printAndExit("Input file not found"); if (!fs.isFile(inFile)) printAndExit("Input should be a file"); if (dfs.exists(outFile)) printAndExit("Output already exists"); // Read from and write to new file FSDataInputStream in = fs.open(inFile); FSDataOutputStream out = dfs.create(outFile); byte buffer[] = new byte[256]; try { System.out.println("Start..."); int bytesRead = 0; while ((bytesRead = in.read(buffer)) > 0) { out.write(buffer, 0, bytesRead); } System.out.println("End....."); } catch (IOException e) { System.out.println("Error while copying file"); } finally { in.close(); out.close(); } } } I wonder whether the rpc port is wrong? or there are other problems? or Someone can give me a example to write to the hdfs? Thanks, Ryan