Any other thoughts regarding that? Separate tool for converting sequence files to plain text would do, but it probably should be re-implemented in C to be as efficient as our current approach. Some time ago I was trying to find such implementation but failed.
Nikita On Fri, Apr 25, 2014 at 11:00 PM, Haohui Mai <h...@hortonworks.com> wrote: > I suggest to build a separate tool streaming from a fs instead of baking it > into webhdfs. One goal in webhdfs is to minimize the dependency of both the > webhdfs server and the client so that other projects easily adopt it. > > If hadoop fs is too slow, it might be a good idea to build a new tool that > reads from stdio, and parses the text format. For example, you can use it > like > > curl http://foo/webhdfs/v1?op=OPEN|parse-text > > ~Haohui > > On Fri, Apr 25, 2014 at 3:16 AM, Nikita Makeev <whale2....@gmail.com> > wrote: > > > Hi. > > > > Sure I will, it will just take some time, as current patch is against > 0.20 > > and 2.0.0 and I'm going to add some more unit tests and make sure the > patch > > conforms to requirements. > > Thanks. > > > > Nikita > > > > > > On Fri, Apr 25, 2014 at 12:17 PM, Akira AJISAKA > > <ajisa...@oss.nttdata.co.jp>wrote: > > > > > # Added hdfs-dev@ > > > > > > Hi Nikita, > > > > > > I'm personally very interested in the functionality! > > > Please create a issue on ASF JIRA and attach your patch. > > > > > > Here is the documentation of creating a patch and attaching it. > > > http://wiki.apache.org/hadoop/HowToContribute > > > > > > Thanks, > > > Akira > > > > > > > > > (2014/04/25 2:09), Nikita Makeev wrote: > > > > > >> In my company we're using webhdfs a lot. One of usage pattern is > > exporting > > >> files from hdfs to external systems which was done before webhdfs by > > >> calling 'hadoop fs -text' to local filesystem and then transferring to > > >> final destination. We're using 'text' because of storing data on hdfs > in > > >> sequence files and our external systems can't read them. We find > webhdfs > > >> very handy, particularly because of very little overhead for starting > up > > >> compared to 'hadoop fs', so we made a patch to make datanode capable > of > > >> doing 'text' via webhdfs. Though I think our usage pattern is hardly > > >> common, I still asking the community, is there any interest in such > > >> functionality. > > >> > > >> SY, > > >> > > >> Nikita Makeev > > >> > > >> > > > > > > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You. >
