I am not familiar with Logstash or Fluentd, but I am familiar with Apache Flume.
Log4j 2 provides capabilities to write directly to a SQL or some NoSQL databases. Some users prefer to do this, but it does have some risk as to what to do if the target destination becomes unavailable. As Remko pointed out, the failover appender can be used to mitigate this somewhat. While we would like all the Appenders to be able to recover for errors connecting to their destinations I am not sure if all of them do at this time. The main advantages you get by using the appenders are simplicity and speed. If they can recover from errors then you would have pretty good reliability, assuming you do something when the connection is unavailable. The reason Log4j comes with a Flume Appender is that Flume can provide more reliability than you will get with the Appenders out of the box. That said, the Flume Appender has 3 variations, 2 of which will involve having log events written to local disk before they are forwarded on to Flume. This same technique could be used with other Appenders to insure that the log events are actually delivered. I should note that the current version of Flume supports the embedded Flume agent, which currently can only write to a remote Flume using Avro. There has been discussion on the Flume mailing list to relax this so in the future it may be possible to have the Flume Appender use the embedded Flume Agent to send data to any of the NoSQL databases Flume supports. Ralph > On Jul 21, 2015, at 11:01 PM, kusmanjali <[email protected]> wrote: > > 1. Most of the logging frame works use a log forwarder + queue > mechanism(Logstash, Fluectd) to store logs into database. What is the > advantage of using this over using Log4j NoSQL appenders to write directly > to the database. > > 2. Any material/link to get more insight into how log4j2 handles the > database connection and failover. And how we can scale this architecture to > store logs from multiple servers into a single database. > > Our idea is to build a central logging system just by using Log4 and doing > away with forwarders and queues. > > -- > with regards > Kusmanjali Jenamoni --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
