Khadar, I am not sure if your reply was meant as a response to my comment or just as additional information. The Log4j 2 FlumeAppender is not part of Flume. See http://logging.apache.org/log4j/2.x/manual/appenders.html#FlumeAvroAppender. However, Log4j 2 is still waiting for its first release so you will have to build it yourself if you want to try it.
Ralph On Jul 27, 2012, at 6:58 AM, khadar basha wrote: > FlumeLog4jAvroAppender is available as part of 0.94. But its not available in > 1.2.Log4jAppender is from flume-ng-log4jappender-1.2.0.jar > > I think it is replace as part of 1.2. > > On Fri, Jul 27, 2012 at 6:31 PM, Ralph Goers <[email protected]> wrote: > You might consider looking at Log4j 2. It has a Flume Appender that records > the whole formatted log message in the body. In addition, it will record the > MDC fields as well. > > Sent from my iPad > > On Jul 27, 2012, at 5:49 AM, khadar basha <[email protected]> wrote: > >> Hi >> >> I am using Flume1.2. Using avro source and hdfs sink. Sending message using >> the Logger channel from application server. For that i am using the >> org.apache.flume.clients.log4jappender.Log4jAppender in MyApp. >> >> But i am getting only body of the message (description). loosing the time, >> thread, Level information. >> >> flume-conf.properties file >> ================== >> agent2Test1.sources = seqGenSrc >> agent2Test1.channels = memoryChannel >> agent2Test1.sinks = loggerSink >> >> # For each one of the sources, the type is defined >> agent2Test1.sources.seqGenSrc.type = avro >> agent2Test1.sources.seqGenSrc.bind=localhost >> agent2Test1.sources.seqGenSrc.port=41414 >> >> # interceptors for host and date >> agent2Test1.sources.seqGenSrc.interceptors = time hostInterceptor >> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.type = >> org.apache.flume.interceptor.HostInterceptor$Builder >> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.hostHeader = host >> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.useIP = false >> agent2Test1.sources.seqGenSrc.interceptors.hostInterceptor.hostHeader.preserveExisting >> = false >> agent2Test1.sources.seqGenSrc.interceptors.time.type = >> org.apache.flume.interceptor.TimestampInterceptor$Builder >> >> # The channel can be defined as follows. >> agent2Test1.sources.seqGenSrc.channels = memoryChannel >> >> # Each sink's type must be defined >> agent2Test1.sinks.loggerSink.type = hdfs >> agent2Test1.sinks.loggerSink.hdfs.path = >> hdfs://hadoopHost:8020/data/%Y/%m/%d/%{host}/Logs >> >> agent2Test1.sinks.loggerSink.hdfs.fileType = DataStream >> >> #Specify the channel the sink should use >> agent2Test1.sinks.loggerSink.channel = memoryChannel >> >> # Each channel's type is defined. >> agent2Test1.channels.memoryChannel.type = memory >> >> # Other config values specific to each type of channel(sink or source) >> # can be defined as well >> # In this case, it specifies the capacity of the memory channel >> agent2Test1.channels.memoryChannel.capacity = 1000 >> >> >> >> Sample java program to generate the log message: >> ===================================== >> >> >> package com.test; >> >> >> import org.apache.flume.clients.log4jappender.Log4jAppender; >> import org.apache.log4j.Logger; >> import org.apache.log4j.MDC; >> import org.apache.log4j.PatternLayout; >> >> import java.util.UUID; >> >> >> public class Main { >> static Logger log = Logger.getLogger(Main.class); >> >> public static void main(String[] args) { >> try { >> Log4jAppender appender = new Log4jAppender(); >> appender.setHostname("localhost"); >> appender.setPort(41414); >> appender.setLayout(new PatternLayout("%d [%c] (%t) <%X{user} >> %X{field}> %m")); >> // appender.setReconnectAttempts(100); >> >> appender.activateOptions(); >> >> log.addAppender(appender); >> >> MDC.put("user", "chris"); >> // while (true) { >> MDC.put("field", UUID.randomUUID().toString()); >> log.info("=====> Hello World"); >> try { >> throw new Exception("Testing"); >> } catch (Exception e) { >> log.error("Gone wrong ===>", e); >> } >> //} >> System.in.read(); >> System.in.read(); >> } >> catch (Exception e) { >> e.printStackTrace(); >> } >> } >> } >> >> >> >> I am missing any config here ? >> >> >> >> -- >> Thanks, >> Khadar >> > > > > -- > Thanks, > Khadar >
