Thanks, Dean, for your ideas. Unfortunately, one of the constraints on our logging solution is to minimize additional network traffic. OutputDebugString( ) is an elegant and efficient single-pc solution, as it uses shared memory (see http://unixwiz.net/techtips/outputdebugstring.html).
I am not sure I agree that the RollingFileAppender approach is so bad. Typically the logs are write-only (nobody has need to look at them), and if they do have need examination, there is a specific time window of interest (just before/after some anomaly). An alternative to having a background process to merging is to instead do it on-demand. This approach would involving writing a simple merge app with GUI that allows the user to specify a time range (and perhaps select which log files) of interest to merge. The other two goals of our logging solution cause minimal performance hit when logging and is reliable. Any other thoughts on this? Thanks again, -Mike ________________________________ From: Dean Fiala [mailto:[EMAIL PROTECTED] Sent: Thursday, February 07, 2008 9:44 PM To: [email protected] Subject: re: Multi-app centralized logging vs. per app logging and background process consolidation The RollingFile merge idea sounds like a lot of extra work and overhead for little obvious gain. The Publish-Subscribe sounds like a much better direction, but if you are putting everything in a central repository, then there is only one subscriber and there is no need for the overhead of who wants what. Just publish everything to the central place. A couple of 2x4 solutions that require little more than configuration... 1) AdoNetAppenders to to log everything to a centralized database 2) SmtpAppenders to log everything to a single email account The advantage of the database is everything is neatly stored in typed columns, timestamped, and easily sortable and analyzed. Triggers and alerts will let you do some fancy things if needed. Lots of built in tools for back-ups, archiving, etc. The disadvantage is that it might not work with your database/server. The application logging load might overwhelm the database. Probably not the best answer if the systems are widely distributed (ie not on the same LAN). You'll probably also want to build an app to look at the log info in, so there will be some build time/cost. The Smtp option isn't as flexible, but gosh it's cheap and easy and works not matter how widely flung the servers are. Throw Outlook (or Outlook Express) on a box and your have a centralized logging client, with a modicum of search and sorting capability already built in. No UI to build. Heck you could even just use a web mail client. Of course, if you need to send each log event a single message you might make your mail server (or administrator) angry, and parsing through the data will require an export and another app. You might also have delivery issues that throw off time synching. Another 2x4 solution is to use LocalSysLogAppender/RemoteSyslogAppender to ultimately get the messages to a single server's syslog service. There are some free/cheap client tools for working with the syslog service. Everything comes in a known log format. There might be some voodoo involved in getting the syslog services all talking together and/or using the RemoteSyslogAppender. If you need to get fancier, an approach based on the RemotingAppender could work. You'd need to build a Remoting server application (hosted as a Windows Service or on IIS), and a client app to query it. If necessary you'll also need to figure out how to persist the log information and build the appropriate goodies to handle all that. This would work on a locally or widely distributed set of servers. One possible issue is that buffering could throw off the time synching . If you have the infrastructure, you might want to check out the MSMQAppender. Would give you very robust message delivery. You'd have to build similar helper apps as you would in the Remoting solution. Fun problem to think about. Good luck. Regards, Dean Fiala ________________________________ A previous generation application suite simply uses OutputDebugString( ) to achieve centralized logging for a multi-app suite running on a single host. The next generation replacement is in the design stage, and will be distributed. It is desired to maintain an efficient and reliable means of centralized logging. One idea is some sort of publish-subscribe model, whereby each app writes to a publisher, with the publisher responsible for delivering the messages to each subscriber. Another idea is to have each app use a RollingFileAppender, and have a background process examine the time tags to assemble a centralized log from the individual files. Obviously time-syncing will be necessary between hosts, and a means to determine any offsets. Both of these approaches may involve more work than we would like to budget for, and may have inherent time-order ambiguity for messages that are close together due to OS context switching. Any comments on these approaches? Any other ideas on how better to do this? Thanks, -Mike
