I just went through a similar process, but using an S3 bucket as the hdfs sink. Checking the collector's log file would be my first suggestion.
On Sat, Feb 25, 2012 at 1:37 PM, alo alt <[email protected]> wrote: > Hi, > > what happen when you start fume with a sink to STDOUT? > > start flume: > flume dump 'syslogTcp(5140)' > > netcat into it: > echo "<37>hello via syslog" | nc localhost 5140 > > - Alex > > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > On Feb 24, 2012, at 4:21 AM, Julien Ricard wrote: > > > Hello, > > > > I'm trying to understand why my collector doesn't write anything to the > sink, file or hdfs destination. > > > > I'm using the cloudera release on debian squeeze. > > > > My setup is really simple. "web7" is an agent, and it sends to > "collector1-flume" which should write to disk or hdfs. The collector is, at > this time, on the same machine than the flume-master node. > > > > At this time, I start the collector with : > > flume node -n collector1-flume & > > > > web7: syslogTcp(5140) | agentSink("collector2-flume",35853); > > collector1-flume : collectorSource(35853) | > collectorSink("file:///tmp","req"); > > > > web7 listens on the port 5140 : > > tcp 0 0 0.0.0.0:5140 0.0.0.0:* > LISTEN > > > > collector1-flume listens on the port 35853 : > > tcp 0 0 0.0.0.0:35853 0.0.0.0:* > LISTEN > > > > The master says that the nodes are alive. The configurations of the > nodes seem to be correct when I read them on the :35862/flumeagent.jsp page. > > > > I tried to listen to the incoming packets with netcat, the syslog is > correctly sending to the agent, and the agent is sending to the collector, > I don't understand what the collector is not doing correctly. > > > > Any idea where to look ? > > > > Thanks ! > >
