Hi Mohit, The prefix in each line of the config file says "foo", and represents the name of the agent. So you would need to specify "--name foo" on the command line so that flume knows which agent to use (because a single config file can actually specify configs for multiple agents). Also, please see the help by running "./bin/flume-ng help". I've posted the output here for your convenience:
Usage: ./bin/flume-ng <command> [options]... commands: help display this help text node run a Flume node avro-client run an avro Flume client global options: --conf,-c <conf> use configs in <conf> directory --classpath,-C <cp> override the classpath for flume jars node options: --conf-file,-f specify a config file (required) --name,-n the name of this node (required) --help,-h display help text avro-client options: --host,-H <host> hostname to which events will be sent (required) --port,-p <port> port of the avro source (required) --filename,-F <file> text file to stream to avro source [default: std input] --help,-h display help text Note that if <conf> directory is specified, then it is always included first in the classpath. So look at "commands". You need to specify exactly one of those... you're specifying "node", as expected. Next, look at "global options"... where global means that these options are relevant whether you choose the "node" command or the "avro-client" command. You specified the --conf option, but the value you specified is not the conf dir. You specified the conf file (which needs to be the value of the --conf-file option). Also, no need to specify --classpath. Next, the "node options" section shows 3 options for when you specify the "node" command. I see that you're missing both "--conf-file" and "--name", both of which are listed as required. No need to look at "avro-client" options for your particular case. So something like this should work: ./bin/flume-ng node --conf ../conf --conf-file ../conf/flume-conf.properties --name foo Cheers, Will On Fri, Mar 30, 2012 at 4:57 PM, Mohit Anchlia <[email protected]>wrote: > I am trying to test how flume ng works so I create a conf and tried to run: > > foo.sources = avroSrc > foo.channels = memoryChannel > foo.sinks = hdfsSink > # For each one of the sources, the type is defined > foo.sources.avroSrc.type = avro > # The channel can be defined as follows. > foo.sources.avroSrc.channels = memoryChannel > # Each sink's type must be defined > foo.sinks.hdfsSink.type = hdfs > foo.sinks.hdfsSink.path = hdfs://dsdb1:9000/flume > #Specify the channel the sink should use > foo.sinks.hdfsSink.channel = memoryChannel > # Each channel's type is defined. > foo.channels.memoryChannel.type = memory > # Other config values specific to each type of channel(sink or source) > # can be defined as well > # In this case, it specifies the capacity of the memory channel > foo.channels.memoryChannel.capacity = 100 > > But java process is not running: > > [root@dslg1 flume-1.2.0-incubating-SNAPSHOT]# ./bin/flume-ng node --conf > ../conf/flume-conf.properties > + exec /usr/java/jdk1.6.0_24/bin/java -Xmx20m -cp > '../conf/flume-conf.properties:/root/.mohit/flume-1.2.0-incubating-SNAPSHOT/lib/*' > -Djava.library.path= org.apache.flume.node.Application > log4j:WARN No appenders could be found for logger > (org.apache.flume.node.Application). > log4j:WARN Please initialize the log4j system properly. > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for > more info. > -- > Can you please suggest what I might be doing wrong? > > >
