RE: File system full causes log4cxx to crash
Probably a limit you can configure with your shell. In bash it's ulimit -c unlimited to get full core dumps and ulimit -a to see the settings (0 bytes by default on my shell.) It and gdb might not be helpful anyway if your log4cxx library is stripped of debugging symbols. If you do pursue getting a core, please don't send it to me :). Install gdb and get the stack trace. We're a month from beta and I still have other bugs to fix, so I'm putting this one to rest for now...
RE: File system full causes log4cxx to crash
With 0.9.7 your app threw and exception and aborted when the log volume filled up. Putting a try/catch around the log call prevented the crash. I put the same try/catch in my wrapper functions and that worked as well, so that's all there was to it. I like simple solutions. Thanks for pointing me in the right direction. -Original Message- From: Peter Steele [mailto:[EMAIL PROTECTED] Sent: Monday, September 22, 2008 5:46 PM To: Log4CXX User Subject: RE: File system full causes log4cxx to crash Okay, thanks, I'll give it a try. -Original Message- From: Jacob L. Anawalt [mailto:[EMAIL PROTECTED] Sent: Monday, September 22, 2008 4:24 PM To: Log4CXX User Subject: Re: File system full causes log4cxx to crash On 2008-09-22 17:03, Peter Steele wrote: We're using 0.9.7 in fact, under FreeBSD. Our case is bit more Maybe we need to catch this IOException in our wrapper functions? The IOException was in the 0.10.x code. I didn't see any exception around the writing to files in 0.9.7. As near as I could tell the iostream library was handing the full filesytem error by not writing any more but the rest of the program and logging kept working. Perhaps it is something with the C wrappers. You could try my test program with a small file based filesystem loop mounted at /mnt -- Jacob Anawalt Gecko Software, Inc. [EMAIL PROTECTED] 435-752-8026
Re: File system full causes log4cxx to crash
Jacob L. Anawalt [EMAIL PROTECTED] writes: I am glad it worked out for you. At the same time I am concerned that you had to do this and I didn't. I would like to reproduce the problem so that I can handle it, preferably without putting try/catch around every logging statement... Some time back, I sent a message to the list asking what the behavior of an appender that throws an exception should be. As it is, a stray exception can unwind all the way through the library into the calling code. To my mind, wrapping log invocations with try/catch blocks isn't a completely satisfactory solution, as if you have multiple appenders associated with a logger, the exception will prevent subsequent appenders from being invoked. Personally, I think log4cxx should provide a guarantee that a log invocation will not throw an exception under any circumstances. One of the common use cases for emitting a log event is when you're in a error handling path, the last thing you want to worry about is the logging framework making hash of things. Two alternatives that come to mind are that every appender's append() method needs to have a try/catch block to squelch any exceptions, or that the appendLoopOnAppenders should have a try/catch block around each call to doAppend() invocation. Or since all it takes is one bad appender (and even if all appenders that are distributed with log4cxx are fixed, there's no guarantee that user's appenders will also be), a hybrid of the two approaches where the try/catch block around doAppend() logs an internal message about the unhandled exception. Basically, a framework shouldn't allow a callback to subvert or compromise the function of the framework itself. --jtc -- J.T. Conklin
Re: File system full causes log4cxx to crash
I agree wholeheartedly If it gets into something like the scenario mentioned below, maybe the logger can put a syslog entry, or console/stderr output (as configured by the user in some global setting)? Renny Koshy [EMAIL PROTECTED] (J.T. Conklin) 09/23/2008 01:12 PM Please respond to Log4CXX User log4cxx-user@logging.apache.org To Log4CXX User log4cxx-user@logging.apache.org cc Subject Re: File system full causes log4cxx to crash Jacob L. Anawalt [EMAIL PROTECTED] writes: I am glad it worked out for you. At the same time I am concerned that you had to do this and I didn't. I would like to reproduce the problem so that I can handle it, preferably without putting try/catch around every logging statement... Some time back, I sent a message to the list asking what the behavior of an appender that throws an exception should be. As it is, a stray exception can unwind all the way through the library into the calling code. To my mind, wrapping log invocations with try/catch blocks isn't a completely satisfactory solution, as if you have multiple appenders associated with a logger, the exception will prevent subsequent appenders from being invoked. Personally, I think log4cxx should provide a guarantee that a log invocation will not throw an exception under any circumstances. One of the common use cases for emitting a log event is when you're in a error handling path, the last thing you want to worry about is the logging framework making hash of things. Two alternatives that come to mind are that every appender's append() method needs to have a try/catch block to squelch any exceptions, or that the appendLoopOnAppenders should have a try/catch block around each call to doAppend() invocation. Or since all it takes is one bad appender (and even if all appenders that are distributed with log4cxx are fixed, there's no guarantee that user's appenders will also be), a hybrid of the two approaches where the try/catch block around doAppend() logs an internal message about the unhandled exception. Basically, a framework shouldn't allow a callback to subvert or compromise the function of the framework itself. --jtc -- J.T. Conklin
RE: File system full causes log4cxx to crash
Peter - To add to your comments: In our systems (real-time call processing engines for carrier networks) a s/w fault is many orders of magnitude worse than no logs being available. One thing that happens in our customer environments is that when something goes wrong... the IT/sysadmin folks will look at syslog entries -- and if the appender or logger framework were to log something they'd see it. The more 'process' oriented companies will have their NOC's review the logs and trap major issues as soon as they happen. Renny Koshy Peter Steele [EMAIL PROTECTED] 09/23/2008 01:26 PM Please respond to Log4CXX User log4cxx-user@logging.apache.org To Log4CXX User log4cxx-user@logging.apache.org cc Subject RE: File system full causes log4cxx to crash I’ll add my own agreement to this. It’s not acceptable for a logger to crash an application when the volume becomes full. My solution to use a try/catch works for us for now since we only have a single appender, but as was mentioned below if we decided to add a second appender the try/catch is a very big hammer, potentially preventing the other appender from running. Still, it’s better than the application crashing on us. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Sent: Tuesday, September 23, 2008 10:20 AM To: Log4CXX User Subject: Re: File system full causes log4cxx to crash I agree wholeheartedly If it gets into something like the scenario mentioned below, maybe the logger can put a syslog entry, or console/stderr output (as configured by the user in some global setting)? Renny Koshy [EMAIL PROTECTED] (J.T. Conklin) 09/23/2008 01:12 PM Please respond to Log4CXX User log4cxx-user@logging.apache.org To Log4CXX User log4cxx-user@logging.apache.org cc Subject Re: File system full causes log4cxx to crash Jacob L. Anawalt [EMAIL PROTECTED] writes: I am glad it worked out for you. At the same time I am concerned that you had to do this and I didn't. I would like to reproduce the problem so that I can handle it, preferably without putting try/catch around every logging statement... Some time back, I sent a message to the list asking what the behavior of an appender that throws an exception should be. As it is, a stray exception can unwind all the way through the library into the calling code. To my mind, wrapping log invocations with try/catch blocks isn't a completely satisfactory solution, as if you have multiple appenders associated with a logger, the exception will prevent subsequent appenders from being invoked. Personally, I think log4cxx should provide a guarantee that a log invocation will not throw an exception under any circumstances. One of the common use cases for emitting a log event is when you're in a error handling path, the last thing you want to worry about is the logging framework making hash of things. Two alternatives that come to mind are that every appender's append() method needs to have a try/catch block to squelch any exceptions, or that the appendLoopOnAppenders should have a try/catch block around each call to doAppend() invocation. Or since all it takes is one bad appender (and even if all appenders that are distributed with log4cxx are fixed, there's no guarantee that user's appenders will also be), a hybrid of the two approaches where the try/catch block around doAppend() logs an internal message about the unhandled exception. Basically, a framework shouldn't allow a callback to subvert or compromise the function of the framework itself. --jtc -- J.T. Conklin
Re: File system full causes log4cxx to crash
On 2008-09-20 11:37, Peter Steele wrote: We've experienced several cases of our logging volume becoming full during the course of execution of our application. What we would like to happen in this case is simply to lose any log further messages that are written after the file system becomes full I wanted to know what would happen to my programs using log4cxx (0.9.7) in this case so I wrote a couple of test apps and created a 1MiB block file and mounted it as a loop device in /mnt. First I tried a simple C app using open and blocking write to see what happens. By picking a buffer size of 8096 I was able to see full writes, a partial write and then an error with perror reporting 'No space left on device'. My code exits the loop at this point. Next I tried a simple C++ log4cxx app that writes to the disk and console via a MyApp logger. After it fill the disk the app continues to work and continues to write to the console. Since the writer is ofstream based, I imagine the bad bit was set and further writes just get skipped. Looking at FileOutputStream::write in svn head I see that if apr_file_write doesn't return APR_SUCCESS, it throws an IOException with the write call's status. I am not familar enough with the code to see who/what should catch that, but to work like 0.9.7 I'd expect it to be caught at some appropriate level to disable that particular appender. (Maybe in WriterAppender::append?) If the exception is not caught perhaps it is the crash you see. -- Jacob Anawalt Gecko Software, Inc. [EMAIL PROTECTED] 435-752-8026
RE: File system full causes log4cxx to crash
Looking at FileOutputStream::write in svn head I see that if apr_file_write doesn't return APR_SUCCESS, it throws an IOException with the write call's status. I am not familar enough with the code to see who/what should catch that, but to work like 0.9.7 I'd expect it to be caught at some appropriate level to disable that particular appender. (Maybe in WriterAppender::append?) If the exception is not caught perhaps it is the crash you see. We're using 0.9.7 in fact, under FreeBSD. Our case is bit more complicated in that we are using log4cxx in a C application, with wrapper functions to let us call the log4cxx functions from our C layer. Maybe we need to catch this IOException in our wrapper functions? I'll have to investigate this. Thanks for the idea. Peter
RE: File system full causes log4cxx to crash
Does not one have any comments on this? Surely someone must have experienced this scenario, specially the volume where logs are being recorded becoming filled up. From: Peter Steele [mailto:[EMAIL PROTECTED] Sent: Friday, September 19, 2008 11:46 AM To: Log4CXX User Subject: File system full causes log4cxx to crash We've experienced several cases of our logging volume becoming full during the course of execution of our application. What we would like to happen in this case is simply to lose any log further messages that are written after the file system becomes full, but what happens instead is log4cxx crashes and brings down our application. We're using the RollingFileAppender on a Unix box. Is there some way to configure log4cxx to behave a little less drastically in a case like this? Ideally, the best behavior in a case like this would be for log4cxx to delete the oldest backup log file and continue logging, but even simply suspending logging would be better than a crash. Any suggestions?
RE: File system full causes log4cxx to crash
Peter - Sorry, been a bit busy lately, and missed this one... I don't believe it's a controllable behavior, from what I recall it is an OS level behavior on what happens when it is out of resources. I've seen apps not using Log4CXX do the same thing. Or worse, I've seen apps get stuck. The only solution we've seen -- as a general case -- is to monitor the volume via SNMP or using a simple script, and notify the admins BEFORE it fails. Regards, Renny Koshy Peter Steele [EMAIL PROTECTED] 09/20/2008 01:37 PM Please respond to Log4CXX User log4cxx-user@logging.apache.org To Log4CXX User log4cxx-user@logging.apache.org cc Subject RE: File system full causes log4cxx to crash Does not one have any comments on this? Surely someone must have experienced this scenario, specially the volume where logs are being recorded becoming filled up. From: Peter Steele [mailto:[EMAIL PROTECTED] Sent: Friday, September 19, 2008 11:46 AM To: Log4CXX User Subject: File system full causes log4cxx to crash We’ve experienced several cases of our logging volume becoming full during the course of execution of our application. What we would like to happen in this case is simply to lose any log further messages that are written after the file system becomes full, but what happens instead is log4cxx crashes and brings down our application. We’re using the RollingFileAppender on a Unix box. Is there some way to configure log4cxx to behave a little less drastically in a case like this? Ideally, the best behavior in a case like this would be for log4cxx to delete the oldest backup log file and continue logging, but even simply suspending logging would be better than a crash. Any suggestions?
