Re: Text cmd in webhdfs

2014-04-28 Thread Nikita Makeev
Any other thoughts regarding that? Separate tool for converting sequence files to plain text would do, but it probably should be re-implemented in C to be as efficient as our current approach. Some time ago I was trying to find such implementation but failed. Nikita On Fri, Apr 25, 2014 at 11:0

Re: Text cmd in webhdfs

2014-04-25 Thread Haohui Mai
I suggest to build a separate tool streaming from a fs instead of baking it into webhdfs. One goal in webhdfs is to minimize the dependency of both the webhdfs server and the client so that other projects easily adopt it. If hadoop fs is too slow, it might be a good idea to build a new tool that r

Re: Text cmd in webhdfs

2014-04-25 Thread Nikita Makeev
Hi. Sure I will, it will just take some time, as current patch is against 0.20 and 2.0.0 and I'm going to add some more unit tests and make sure the patch conforms to requirements. Thanks. Nikita On Fri, Apr 25, 2014 at 12:17 PM, Akira AJISAKA wrote: > # Added hdfs-dev@ > > Hi Nikita, > > I'm

Re: Text cmd in webhdfs

2014-04-25 Thread Akira AJISAKA
# Added hdfs-dev@ Hi Nikita, I'm personally very interested in the functionality! Please create a issue on ASF JIRA and attach your patch. Here is the documentation of creating a patch and attaching it. http://wiki.apache.org/hadoop/HowToContribute Thanks, Akira (2014/04/25 2:09), Nikita Make

Text cmd in webhdfs

2014-04-24 Thread Nikita Makeev
In my company we're using webhdfs a lot. One of usage pattern is exporting files from hdfs to external systems which was done before webhdfs by calling 'hadoop fs -text' to local filesystem and then transferring to final destination. We're using 'text' because of storing data on hdfs in sequence fi