Hi I am working on hdfs in java and want to append data to an existing
file. I am using hadoop 2.2.0 and searched a lot about it but found nothing.

I know that if I want to append data then i can first copy data to a
temporary file and then write it back to the original file with appended
data. But my file data is of millions of lines. So each time performing
this operation will be very expensive for me.

Is there any built in functionality for appending data to hdfs file in
hadoop using java?

Currently I am using following piece of code, but its not efficient.

BufferedReader bfr=new BufferedReader(new
InputStreamReader(hdfs.open(newFilePath)));     //open file first
            String str = null;
            BufferedWriter br=new BufferedWriter(new
OutputStreamWriter(hdfs.create(newFilePath,true)));
            while ((str = bfr.readLine())!= null)
            {
                br.write(str); // write file content
                br.newLine();
                //System.out.println("   ->>>>>  "+str);
            }
            br.write(name);  // append into file
            br.newLine();
            br.close(); // close it

Any efficient idea about this?

Regards
Jamil Shehzad

Reply via email to