Hello, Not too sure about Flume OG - I barely used it for a week before shifting to NG. Would there be any particular reason you are using OG?
Yes, you can take a look at SyslogTcpSource (in NG), https://people.apache.org/~mpercy/flume/flume-1.2.0-incubating-SNAPSHOT/docs/apidocs/org/apache/flume/source/SyslogTcpSource.html This can be modified according to this Netty example (for asynchronous large file streaming) : http://static.netty.io/3.5/xref/org/jboss/netty/example/http/file/package-summary.html This is pretty much what I did when I needed a Flume HTTP source to match my requirements. Regards, Harish On Tue, Jul 24, 2012 at 3:02 AM, Chandrashekhar Shaw <[email protected]>wrote: > I am using Flume - OG > https://github.com/downloads/cloudera/flume/flume-distribution-0.9.4-bin.tar.gz > . > > It there an available version which implements netty?? > > Thanks > Chandrashekhar Shaw > > > On Jul 23, 2012, at 8:00 PM, Harish Mandala wrote: > > Hi, > > Which one of the sources are you using? Maybe a flume source that > implements something like > http://docs.jboss.org/netty/3.2/xref/org/jboss/netty could be useful. > > Regards, > Harish > > > On Mon, Jul 23, 2012 at 7:52 AM, Chandrashekhar Shaw > <[email protected]>wrote: > > Hi, > > > I am using flume for transferring logs from 20 agent boxes to 1 collector > > box. The file size which I am trying to transfer is of about 1 GB per hour. > > So every hour my agents run to transfer the log file generated during the > > last hour. > > I am using DFO sink for file transfer. > > > But then I am facing following problems: > > > 1) Some of the agents are not able to transfer the whole file even in one > > hour and the processes run continuously for hours. > > 2) Some of the agents stop within 5-6 minutes without sending the file > > completely. > > > I want to know > > > 1) if somebody else has faced the same issues? > > 2) What could be the problem? > > > Thanks > > Chandrashekhar Shaw > > > > > >
