For testing purposes, I tried with the following configuration without much luck. I see that the process started fine but it just does not write anything to the sink. I guess i am missing something here. Can one of you gurus take a look and suggest what i am doing wrong?
Thanks, Bhaskar agent1.sources = tail agent1.channels = MemoryChannel-2 agent1.sinks = svc_0_sink agent1.sources.tail.type = exec agent1.sources.tail.command = tail -f /var/log/access.log agent1.sources.tail.channels = MemoryChannel-2 agent1.sinks.svc_0_sink.type = FILE_ROLL agent1.sinks.svc_0_sink.sink.directory=/flume_runtime/logs agent1.sinks.svc_0_sink.rollInterval=0 agent1.channels.MemoryChannel-2.type = memory On Thu, Jun 14, 2012 at 4:26 AM, Guillaume Polaert <gpola...@cyres.fr>wrote: > Hi Bhaskar, > > This is the flume.conf (http://pastebin.com/WULgUuaf) what I'm using. > I have an avro server on the hadoop-m host and one agent per node (slave > hosts). Each agent send the ouput of a exec command to avro server. > > Host1 : exec -> memory -> avro (sink) > > Host2 : exec -> memory -> avro > >>>>> MainHost : avro > (source) -> memory -> rolling file (local FS) > ... > > Host3 : exec -> memory -> avro > > > Use your own exec command to read Apache log. > > Guillaume Polaert | Cyrès Conseil > > De : Bhaskar [mailto:bmar...@gmail.com] > Envoyé : mercredi 13 juin 2012 19:16 > À : flume-user@incubator.apache.org > Objet : Newbee question about flume 1.2 set up > > Good Afternoon, > I am a newbee to flume and read thru limited documentation available. I > would like to set up the following to test out. > > 1. Read apache access logs (as source) > 2. Use memory channel > 3. Write it to a NFS (or even local) file system > > Can some one help me with the necessary configuration. I am having > difficult time to glean that information from available documentation. I > am sure someone has done such test before and i appreciate if you can pass > on that information. Secondly, I also would like to stream the logs to a > remote server. Is that a log4j configuration or do i need to run an agent > on each host to do so? Any configuration examples would be of great help. > > Thanks, > Bhaskar >
