You need to use Flume’s client API to send data to the Avro Source. Use the thrift source, and generate thrift client for Python using this IDL file: https://github.com/apache/flume/blob/trunk/flume-ng-sdk/src/main/thrift/flume.thrift <https://github.com/apache/flume/blob/trunk/flume-ng-sdk/src/main/thrift/flume.thrift>
You can use that to send data to the thrift source at that point. Thanks, Hari Shreedharan > On May 13, 2015, at 8:06 PM, 鹰 <[email protected]> wrote: > > I send data by python scripts use socket send the code like this: > > import sys > from socket import * > > > HOST = '192.168.1.117' > > > > PORT =44444 > BUFSIZ = 1024 > ADDR = (HOST, PORT) > > tcpCliSock = socket(AF_INET, SOCK_STREAM) > tcpCliSock.connect(ADDR) > i=0 > for x in range(3): > print x, "xx" > n=tcpCliSock.send("test datas from flume") > tcpCliSock.close() > > > > ------------------ 原始邮件 ------------------ > 发件人: "Hari Shreedharan";<[email protected]>; > 发送时间: 2015年5月14日(星期四) 上午10:53 > 收件人: "[email protected]"<[email protected]>; > 主题: Re: set flume send logs to hdfs error > > How are you sending data to the Avro Source? > > > Thanks, > Hari > > On Wed, May 13, 2015 at 7:38 PM, 鹰 <[email protected] > <mailto:[email protected]>> wrote: > hi all , > i'm want set flume send data to hdfs my configure file is lile this : > tier1.sources=source1 > tier1.channels=channel1 > tier1.sinks=sink1 > > tier1.sources.source1.type=avro > tier1.sources.source1.bind=0.0.0.0 > tier1.sources.source1.port=44444 > tier1.sources.source1.channels=channel1 > > tier1.channels.channel1.type=memory > tier1.channels.channel1.capacity=10000 > tier1.channels.channel1.transactionCapacity=1000 > tier1.channels.channel1.keep-alive=30 > > tier1.sinks.sink1.type=hdfs > tier1.sinks.sink1.channel=channel1 > tier1.sinks.sink1.hdfs.path=hdfs://hadoop-home.com:9000/user/hadoop/ > <http://hadoop-home.com:9000/user/hadoop/> > tier1.sinks.sink1.hdfs.fileType=DataStream > tier1.sinks.sink1.hdfs.writeFormat=Text > tier1.sinks.sink1.hdfs.rollInterval=0 > tier1.sinks.sink1.hdfs.rollSize=10240 > tier1.sinks.sink1.hdfs.rollCount=0 > tier1.sinks.sink1.hdfs.idleTimeout=60 > > when I start the flume by this configure file and send data to the port 44444 > I get an error : > org.apache.avro.AvroRuntimeException: Excessively large list allocation > request detected: 154218761 items! Connection closed; > dose anybody can help me ,thanks. >
