My current project may be of interest to you. I have a Java App which runs about 600 threads logging to files using the RollingFileAppender.
Now this app under heavy load writes out about 600 MB of logging every 4 hours when configured at DEBUG. This app is deployed on a four - core dual processor running Linux. Under these conditions the app peaks at a CPU usage of about 10%. When I change the level to WARN I hardly see any change in the CPU profile, so logging forms a very small percentage of the load. What is the projected amount of logging that you will be doing. Unless you are writing GB's every hour I think log4j should be able to handle it. Cheers, Philip. -----Original Message----- From: Paul Duffy [mailto:[EMAIL PROTECTED] Sent: Thursday, August 02, 2007 11:02 AM To: Log4J Users List Subject: [BULK] Re: Looking for high speed binary appender? Importance: Low Folks, To clarify... - my team's current app (highly distributed, born in 98) is using legacy home grown text based logging, - we're looking at an upgrade to log4j. - but major customers are saying they want to increase log column, from (essentially moving from WARN to INFO level). I do not have specific volume info, only that my gut tells he we'll be heading into territory that log4j may not be able to handle. There is talk within my team of going to a more binary based logging to cut down text manipulation costs, disk space, etc. In essence, defer message formatting overhead from runtime to post processing. Obviously we prefer to use any robust open source solutions available. Then there is this from the NetLogger folks... http://dsd.lbl.gov/publications/HPDC02-HP-monitoring.pdf ...which describes a binary based logging framework which seriously outperforms log4j. So my questions are - has anyone built a high volume (binary or otherwise) optimized logging facility atop log4j? - what other solutions have been applied when log4j performance is an issue? Cheers Paul Smith wrote: > > On 02/08/2007, at 9:23 AM, Paul Duffy wrote: > >> Folks, >> >> My team is looking at log4j as a next generation logging option, but >> we have a specific customer need to do high volume logging which a >> text based system may not support. Is anyone aware of a high speed >> binary logging capability that builds upon the log4j infrastructure >> (or otherwise? >> > > You'd be surprised how many events a file-based appender can write. > Can you provide some #'s on how many events you think you'll need to > support? > > Paul > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] __________________ IMPORTANT NOTICE Email from TSI Terminal Systems Inc. (TSI) is confidential and may be legally privileged. If it is not intended for you, please delete it immediately unread. Under no circumstances should this e-mail be redistributed without prior written consent of TSI. The internet cannot guarantee that this communication is free of viruses, interception or interference and anyone who communicates with us by email is taken to accept the risks in doing so. Without limitation, TSI and its affiliates accept no liability whatsoever and howsoever arising in connection with the use of this email. Under no circumstances shall this email constitute a binding agreement for provision of services by TSI, which is subject to the terms and conditions of TSI's standard schedule for Terminal Tariffs which is also available at http://www.tsi.bc.ca. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
