I suggest to build a separate tool streaming from a fs instead of baking it into webhdfs. One goal in webhdfs is to minimize the dependency of both the webhdfs server and the client so that other projects easily adopt it.
If hadoop fs is too slow, it might be a good idea to build a new tool that reads from stdio, and parses the text format. For example, you can use it like curl http://foo/webhdfs/v1?op=OPEN|parse-text ~Haohui On Fri, Apr 25, 2014 at 3:16 AM, Nikita Makeev <whale2....@gmail.com> wrote: > Hi. > > Sure I will, it will just take some time, as current patch is against 0.20 > and 2.0.0 and I'm going to add some more unit tests and make sure the patch > conforms to requirements. > Thanks. > > Nikita > > > On Fri, Apr 25, 2014 at 12:17 PM, Akira AJISAKA > <ajisa...@oss.nttdata.co.jp>wrote: > > > # Added hdfs-dev@ > > > > Hi Nikita, > > > > I'm personally very interested in the functionality! > > Please create a issue on ASF JIRA and attach your patch. > > > > Here is the documentation of creating a patch and attaching it. > > http://wiki.apache.org/hadoop/HowToContribute > > > > Thanks, > > Akira > > > > > > (2014/04/25 2:09), Nikita Makeev wrote: > > > >> In my company we're using webhdfs a lot. One of usage pattern is > exporting > >> files from hdfs to external systems which was done before webhdfs by > >> calling 'hadoop fs -text' to local filesystem and then transferring to > >> final destination. We're using 'text' because of storing data on hdfs in > >> sequence files and our external systems can't read them. We find webhdfs > >> very handy, particularly because of very little overhead for starting up > >> compared to 'hadoop fs', so we made a patch to make datanode capable of > >> doing 'text' via webhdfs. Though I think our usage pattern is hardly > >> common, I still asking the community, is there any interest in such > >> functionality. > >> > >> SY, > >> > >> Nikita Makeev > >> > >> > > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
