Hello Bjoern, Thrift works with binary data - at least with hbase. I have a C# app that people can use to put binary (and get) files in hbase via Thrift. I'll send example code later. I also have java apps that upload files to hdfs directly and are not on the server - but they do need access to the copies of the config files. But they just use the standard java hdfs api.
Best, -stu ------Original Message------ From: Bjoern Schiessle To: hdfs-user@hadoop.apache.org ReplyTo: hdfs-user@hadoop.apache.org Subject: Re: Best way to write files to hdfs (from a Python app) Sent: Aug 10, 2010 05:06 Hi Philip, On Mon, 9 Aug 2010 16:35:07 -0700 Philip Zeyliger wrote: > To give you an example of how this may be done, HUE, under the covers, > pipes your data to 'bin/hadoop fs -Dhadoop.job.ugi=user,group put - > path'. (That's from memory, but it's approximately right; the full > python code is at > http://github.com/cloudera/hue/blob/master/desktop/libs/hadoop/src/hadoop/fs/hadoopfs.py#L692 > ) Thank you! If I understand it correctly this only works if my python app runs on the same server as hadoop, right? I would like to run the python app on a different server. Therefore my two ideas (1) Thrift or (2) hdfs-fuse. Thrift seems to be able to store string content only to hdfs but no binary files. At least I couldn't find an interface for a simple put operation. So at the moment I'm not sure how to continue. Any ideas? Thanks, Björn
