Hi I am working on hdfs in java and want to append data to an existing
file. I am using hadoop 2.2.0 and searched a lot about it but found nothing.
I know that if I want to append data then i can first copy data to a
temporary file and then write it back to the original file with appended
data. But my file data is of millions of lines. So each time performing
this operation will be very expensive for me.
Is there any built in functionality for appending data to hdfs file in
hadoop using java?
Currently I am using following piece of code, but its not efficient.
BufferedReader bfr=new BufferedReader(new
InputStreamReader(hdfs.open(newFilePath))); //open file first
String str = null;
BufferedWriter br=new BufferedWriter(new
OutputStreamWriter(hdfs.create(newFilePath,true)));
while ((str = bfr.readLine())!= null)
{
br.write(str); // write file content
br.newLine();
//System.out.println(" ->>>>> "+str);
}
br.write(name); // append into file
br.newLine();
br.close(); // close it
Any efficient idea about this?
Regards
Jamil Shehzad