[ https://issues.apache.org/jira/browse/HADOOP-1700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ruyue Ma updated HADOOP-1700: ----------------------------- Affects Version/s: 0.15.1 Status: Patch Available (was: Open) this patch gives a simple append implementation based on 0.15.1 version. it is completely compatible with the 0.15.1. It adds a new open() function in DFSClient class. Usage: 1. open for read InputStream is = (FSDataInputStream)dfsclient.open(path, buffersize, "r"); 2. open for append OutputStream os = (FSDataOutputStream)dfsclient.open(path, buffersize, "a"); 3. open for append (if no file, create it) OutputStream os = (FSDataOutputStream)dfsclient.open(path, buffersize, "a+c"); [ Show » ] Ruyue Ma - 25/Dec/07 06:49 PM this patch gives a simple append implementation based on 0.15.1 version. it is completely compatible with the 0.15.1. It adds a new open() function in DFSClient class. Usage: 1. open for read InputStream is = (FSDataInputStream)dfsclient.open(path, buffersize, "r"); 2. open for append OutputStream os = (FSDataOutputStream)dfsclient.open(path, buffersize, "a"); 3. open for append (if no file, create it) OutputStream os = (FSDataOutputStream)dfsclient.open(path, buffersize, "a+c"); > Append to files in HDFS > ----------------------- > > Key: HADOOP-1700 > URL: https://issues.apache.org/jira/browse/HADOOP-1700 > Project: Hadoop > Issue Type: New Feature > Components: dfs > Affects Versions: 0.15.1 > Reporter: stack > Attachments: append.patch, Appends.doc, Appends.doc, Appends.html > > > Request for being able to append to files in HDFS has been raised a couple of > times on the list of late. For one example, see > http://www.nabble.com/HDFS%2C-appending-writes-status-tf3848237.html#a10916193. > Other mail describes folks' workarounds because this feature is lacking: > e.g. http://www.nabble.com/Loading-data-into-HDFS-tf4200003.html#a12039480 > (Later on this thread, Jim Kellerman re-raises the HBase need of this > feature). HADOOP-337 'DFS files should be appendable' makes mention of file > append but it was opened early in the life of HDFS when the focus was more on > implementing the basics rather than adding new features. Interest fizzled. > Because HADOOP-337 is also a bit of a grab-bag -- it includes truncation and > being able to concurrently read/write -- rather than try and breathe new life > into HADOOP-337, instead, here is a new issue focused on file append. > Ultimately, being able to do as the google GFS paper describes -- having > multiple concurrent clients making 'Atomic Record Append' to a single file > would be sweet but at least for a first cut at this feature, IMO, a single > client appending to a single HDFS file letting the application manage the > access would be sufficent. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.