3 Things; 1. Wrong mailing list
2. You might want to consider using hbase or accumulo instead of raw HDFS files 3. It was not hard for me to google “hadoop 2.2.0 append” to find the API calls and configuration settings needed From: Jamil Shehzad [mailto:[email protected]] Sent: Thursday, October 30, 2014 2:14 PM To: [email protected] Subject: Append hdfs file in Java Hi I am working on hdfs in java and want to append data to an existing file. I am using hadoop 2.2.0 and searched a lot about it but found nothing. I know that if I want to append data then i can first copy data to a temporary file and then write it back to the original file with appended data. But my file data is of millions of lines. So each time performing this operation will be very expensive for me. Is there any built in functionality for appending data to hdfs file in hadoop using java? Currently I am using following piece of code, but its not efficient. BufferedReader bfr=new BufferedReader(new InputStreamReader(hdfs.open(newFilePath))); //open file first String str = null; BufferedWriter br=new BufferedWriter(new OutputStreamWriter(hdfs.create(newFilePath,true))); while ((str = bfr.readLine())!= null) { br.write(str); // write file content br.newLine(); //System.out.println(" ->>>>> "+str); } br.write(name); // append into file br.newLine(); br.close(); // close it Any efficient idea about this? Regards Jamil Shehzad
