Hi,
sent via my mobile device On Apr 1, 2012, at 4:44 PM, Mohit Anchlia <[email protected]> wrote: inline > Thanks for the post. > > I have some questions: > > 1. Is writing to the SequenceFile better? I thought sequenceFile is generally > good for binary data. Yes. > > 2. Generally is one agent sufficient for one client? If not then idea is to > just start multiple agents? Yes, put them into the same channel > > 3. How to deal with failures of agents or clients? Use a couple of agents, and in front if them an loadbalancer as example. best, - Alex > > On Sun, Apr 1, 2012 at 2:39 AM, alo alt <[email protected]> wrote: > I wrote a couple of weeks ago a blog post: > http://mapredit.blogspot.de/2012/03/flumeng-evolution.html > > here u found all what you need. > > - Alex > > -- > Alexander Lorenz > http://mapredit.blogspot.com > > On Apr 1, 2012, at 8:26 AM, shekhar sharma wrote: > > > Hello Mohit, > > Create a property file "foo.properties" and then run the following command > > to start your agent > > ./flume-ng node --conf ../conf/ -f ../conf/foo.properties -n foo > > > > and then you can run ur avro client to send some sample data which will be > > collected by the foo's source an it agent i.e. avro source which then it > > will send to HDFS. > > > > flume-ng avro-client --conf ../conf -H localhost -p 41414 -F /etc/passwd > > > > > > you need to define the host and the port for foo.source in your > > foo.properties file. Hope that helps. > > > > Regards, > > Som > > > > > > On Sat, Mar 31, 2012 at 5:27 AM, Mohit Anchlia <[email protected]> > > wrote: > > I am trying to test how flume ng works so I create a conf and tried to run: > > > > foo.sources = avroSrc > > foo.channels = memoryChannel > > foo.sinks = hdfsSink > > # For each one of the sources, the type is defined > > foo.sources.avroSrc.type = avro > > # The channel can be defined as follows. > > foo.sources.avroSrc.channels = memoryChannel > > # Each sink's type must be defined > > foo.sinks.hdfsSink.type = hdfs > > foo.sinks.hdfsSink.path = hdfs://dsdb1:9000/flume > > #Specify the channel the sink should use > > foo.sinks.hdfsSink.channel = memoryChannel > > # Each channel's type is defined. > > foo.channels.memoryChannel.type = memory > > # Other config values specific to each type of channel(sink or source) > > # can be defined as well > > # In this case, it specifies the capacity of the memory channel > > foo.channels.memoryChannel.capacity = 100 > > > > But java process is not running: > > > > [root@dslg1 flume-1.2.0-incubating-SNAPSHOT]# ./bin/flume-ng node --conf > > ../conf/flume-conf.properties > > + exec /usr/java/jdk1.6.0_24/bin/java -Xmx20m -cp > > '../conf/flume-conf.properties:/root/.mohit/flume-1.2.0-incubating-SNAPSHOT/lib/*' > > -Djava.library.path= org.apache.flume.node.Application > > log4j:WARN No appenders could be found for logger > > (org.apache.flume.node.Application). > > log4j:WARN Please initialize the log4j system properly. > > log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for > > more info. > > -- > > Can you please suggest what I might be doing wrong? > > > > > > > >
