Hi, I am not familiar with Logstash and Fluectd, but in general I would assume that most databases may not be able to keep up with log events during a burst, and these mechanisms provide a persisted queue where your log events can be recovered from in case the process or box crashed before all events are saved to the database.
If you have a NonSQL database that is as fast as the intermediate storage then I don't see any advantage to the intermediate storage. But again, I'm not familiar with these products so I may be missing something. 2. Log4j has a failover appender (which must be up and available when log4j starts up). If appending to the main appender fails Log4j will append the event to the failover appender. I would encourage you to study the documentation and experiment with some common failure scenarios. Note that the failover appender never "becomes primary": every event is first attempted to append to the primary appender, and if an exception occurs it is sent to the failover appender. It may be better to rely on the clustering/failover capabilities of your database. About scalability: this depends on (a) how many transactions per second can your database handle, and (b) how many log events per second you expect your application to send during bursts and how many applications you have running simultaneously. If one app gets busy, do they all get busy or is the load independent? If during peaks the apps generate more transactions than the database can handle, then having an intermediate queue may still be valuable. Remko Sent from my iPhone > On 2015/07/22, at 15:01, kusmanjali <[email protected]> wrote: > > 1. Most of the logging frame works use a log forwarder + queue > mechanism(Logstash, Fluectd) to store logs into database. What is the > advantage of using this over using Log4j NoSQL appenders to write directly > to the database. > > 2. Any material/link to get more insight into how log4j2 handles the > database connection and failover. And how we can scale this architecture to > store logs from multiple servers into a single database. > > Our idea is to build a central logging system just by using Log4 and doing > away with forwarders and queues. > > -- > with regards > Kusmanjali Jenamoni --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
