Hello log4j users/developers, I have created a new appender called *EventConsolidatingAppender*. The purpose of this appender is to consolidate multiple events that are received by a single logger within a specified number of seconds into a single event; this single consolidated event is then forwarded to a "downstream" appender.
EventConsolidatingAppender is probably most useful when the downstream appender is SMTPAppender. In this case, you can use EventConsolidatingAppender so that if, say, 200 errors are raised to a given logger within 30 seconds, instead of receiving 200 emails, you'll receive * one* email that contains the messages from all 200 errors. Example use case: Our system makes high-volume, real-time requests to a data provider's web services. When we have a problem contacting a provider's service, we'll raise a warning or error event to a logger. When a data provider's service goes down for a several minutes, we may make multiple hundreds of requests to that service that will all fail. We like using SMTPAppender so that we receive immediate notification of any warnings and errors via email, but if a data provider is down and we're sending an email per error, our email system may get overwhelmed and/or disabled by our hosting provider (Gmail). Using EventConsolidatingAppender with SMTPAppender as the downstream appender solves this problem. This problem probably could also have been solved via the configuration of more sophisticated operations/monitoring software (Nagios, Zenoss, etc.), or via the delivery of events to a JMS queue, but in our case it wasn't desirable to install, configure and maintain something so heavyweight. This appender is thread-safe and implements internal "per-logger" synchronization (since it performs "per-logger" caching). Sample configuration (includes SMTPAppender configuration just for completeness): log4j.appender.ConsolidatingAppender= org.mattklein.log4j.EventConsolidatingAppender log4j.appender.ConsolidatingAppender.threshold=WARN log4j.appender.ConsolidatingAppender.downstreamAppender=EmailAppender log4j.appender.ConsolidatingAppender.delaySecs=30 log4j.appender.EmailAppender=org.apache.log4j.net.SMTPAppender log4j.appender.EmailAppender.threshold=WARN log4j.appender.EmailAppender.layout=org.apache.log4j.PatternLayout log4j.appender.EmailAppender.layout.ConversionPattern=[%d{ISO8601}] [%t] [%c] [%l] %n%p -%m%n log4j.appender.EmailAppender.BufferSize=1 log4j.appender.EmailAppender.SMTPHost=smtp.gmail.com log4j.appender.EmailAppender.SMTPUsername=o...@mattklein.org log4j.appender.EmailAppender.SMTPPassword=password log4j.appender.EmailAppender.From=Ops <o...@mattklein.org> log4j.appender.EmailAppender.To=o...@mattklein.org log4j.appender.EmailAppender.Subject=[ALERT] Warning or error raised # Because we want the SMTPAppender to send emails on WARNINGs as well as ERRORs, we need to provide a custom evaluatorClass log4j.appender.EmailAppender.evaluatorClass= org.mattklein.log4j.MyTriggeringEventEvaluator Sample "consolidated" event: [2011-02-21 10:55:15,901] ERROR - The following 6 events were consolidated since they occurred within 30 seconds of each other Event 1: [2011-02-21 10:54:45,896] WARN - Warn message Event 2: [2011-02-21 10:54:45,898] ERROR - Error message Event 3: [2011-02-21 10:54:57,903] WARN - Warn message Event 4: [2011-02-21 10:54:57,903] ERROR - Error message Event 5: [2011-02-21 10:55:09,905] WARN - Warn message Event 6: [2011-02-21 10:55:09,905] ERROR - Error message The priority of the consolidated event is set to the highest priority of any of the constituent events (i.e., WARN and ERROR events consolidate into an ERROR event). If only a single event is raised within the delaySecs parameter, it will be delivered unmodified to the downstream appender. Note that the delivery of events to your downstream appender will be delayed by up to delaySecs seconds. So, if only a single event is raised and delaySecs is 30, the email for it will be sent with a 30 second lag from the time the event was raised (as the EventConsolidatingAppender waits 30 seconds to see if any other events are raised). We consider this delay to be a fine tradeoff though. If anyone else is interested in this code I'm happy to provide it. Feel free to contact me at mpkl...@gmail.com. Matt Klein