The next release of Flume-1.3.0 adds support for an HTTP source, which will allow you to send data to Flume via HTTP/JSON(the representation of the data is pluggable - but a JSON representation is default). You could use this to write data to Flume from Python, which I believe has good http and json support.
Thanks, Hari -- Hari Shreedharan On Thursday, November 8, 2012 at 10:45 AM, Bart Verwilst wrote: > Hi, > > I've been spending quite a few hours trying to push avro data to Flume > so i can store it on HDFS, this all with Python. > It seems like something that is impossible for now, since the only way > to push avro data to Flume is by the use of deprecated thrift binding > that look pretty cumbersome to get working. > I would like to know what's the best way to import avro data into Flume > with Python? Maybe Flume isnt the right tool and I should use something > else? My goal is to have multiple python workers pushing data to HDFS > which ( by means of Flume in this case ) consolidates this all in 1 file > there. > > Any thoughts? > > Thanks! > > Bart
