Curt, I have been able to reproduce this problem using the latest tar ball at http://littletux.homelinux.org/log4cxx/log4cxx-0.9.8.tar.gz
I used this because it was more convenient for me than trying to figure out how to grab the SVN HEAD. Is this ok? On running against this latest snapshot, I have run onto at least 3 separate segmentation faults at different points in the code. I suspect there is some racing condition because it is unpredictable which of these, if any, I will hit on a given run. I wonder if this is something to do with being on an SMP box. How should I proceed? Separate bug reports for each problem that I have found? Where is the proper place to submit bug reports? This is a pretty big issue for me right now, so I will be continuing to test/debug and report my findings. So far, I have only been able to capture one of these segmentation fault paths using valgrind. I am including the output here. My platform is: OS: RHEL 3 Compiler: gcc version 3.2.3 20030502 Processor Type: I don't have direct access to the machine, so I'm not sure how to determine the processor type, but it is something of the i686 variety. Number of Processors: 2 Some additional info: [ewyles@<host> src]$ uname -a Linux <host> 2.4.21-27.0.2.ELsmp #1 SMP Wed Jan 12 23:35:44 EST 2005 i686 i686 i386 GNU/Linux Here what I have from valgrind. I will continue trying to recreate as many scenarios as possible through valgrind: ==6516== Invalid read of size 4 ==6516== at 0x1BB4B7CC: pthread_mutex_lock (in /lib/tls/libpthread-0.60.so) ==6516== by 0x1BABF1BE: apr_thread_mutex_lock (thread_mutex.c:92) ==6516== by 0x1BA75520: log4cxx::helpers::synchronized::synchronized(log4cxx::helpers::Mutex const&) (synchronized.cpp:30) ==6516== by 0x1BA7E7A6: log4cxx::AppenderSkeleton::doAppend(log4cxx::helpers::ObjectPtrT<log4cxx ::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderskeleton.cpp:93) ==6516== by 0x1BA980EF: log4cxx::helpers::AppenderAttachableImpl::appendLoopOnAppenders(log4cxx: :helpers::ObjectPtrT<log4cxx::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderattachableimpl.cpp:61) ==6516== by 0x1BA858BA: log4cxx::Logger::callAppenders(log4cxx::helpers::ObjectPtrT<log4cxx::spi ::LoggingEvent> const&, log4cxx::helpers::Pool&) (logger.cpp:78) ==6516== by 0x1BA85F3A: log4cxx::Logger::forcedLog(log4cxx::helpers::ObjectPtrT<log4cxx::Level> const&, std::string const&) (logger.cpp:129) ==6516== by 0x1BA880E2: log4cxx::Logger::debug(std::string const&) (logger.cpp:591) ==6516== by 0x8053057: LoggerThread::meth_0() (loggerthread.cpp:36) ==6516== by 0x804F904: LoggerThread::run() (loggerthread.cpp:8032) ==6516== by 0x804F7CE: LoggerThread::runStatic(void*) (loggerthread.cpp:9036) ==6516== by 0x1BB49DEB: start_thread (in /lib/tls/libpthread-0.60.so) ==6516== Address 0x1BD9F934 is 556 bytes inside a block of size 8192 free'd ==6516== at 0x1B903A5D: free (vg_replace_malloc.c:152) ==6516== by 0x1BABD505: apr_allocator_destroy (apr_pools.c:111) ==6516== by 0x1BABDB92: apr_pool_destroy (apr_pools.c:777) ==6516== by 0x1BABD783: apr_pool_terminate (apr_pools.c:568) ==6516== by 0x1BABECAA: apr_terminate (start.c:82) ==6516== by 0x1BA5CABB: log4cxx::helpers::APRInitializer::~APRInitializer() (aprinitializer.cpp:47) ==6516== by 0x1BA5CBEF: __tcf_0 (aprinitializer.cpp:53) ==6516== by 0x1BC63967: __cxa_finalize (in /lib/tls/libc-2.3.2.so) ==6516== by 0x1B9DFED8: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1BACFB19: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1B8F0EDC: _dl_fini (in /lib/ld-2.3.2.so) ==6516== by 0x1BC636D2: exit (in /lib/tls/libc-2.3.2.so) ==6516== ==6516== Invalid write of size 4 ==6516== at 0x1BB4B7D2: pthread_mutex_lock (in /lib/tls/libpthread-0.60.so) ==6516== by 0x1BABF1BE: apr_thread_mutex_lock (thread_mutex.c:92) ==6516== by 0x1BA75520: log4cxx::helpers::synchronized::synchronized(log4cxx::helpers::Mutex const&) (synchronized.cpp:30) ==6516== by 0x1BA7E7A6: log4cxx::AppenderSkeleton::doAppend(log4cxx::helpers::ObjectPtrT<log4cxx ::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderskeleton.cpp:93) ==6516== by 0x1BA980EF: log4cxx::helpers::AppenderAttachableImpl::appendLoopOnAppenders(log4cxx: :helpers::ObjectPtrT<log4cxx::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderattachableimpl.cpp:61) ==6516== by 0x1BA858BA: log4cxx::Logger::callAppenders(log4cxx::helpers::ObjectPtrT<log4cxx::spi ::LoggingEvent> const&, log4cxx::helpers::Pool&) (logger.cpp:78) ==6516== by 0x1BA85F3A: log4cxx::Logger::forcedLog(log4cxx::helpers::ObjectPtrT<log4cxx::Level> const&, std::string const&) (logger.cpp:129) ==6516== by 0x1BA880E2: log4cxx::Logger::debug(std::string const&) (logger.cpp:591) ==6516== by 0x8053057: LoggerThread::meth_0() (loggerthread.cpp:36) ==6516== by 0x804F904: LoggerThread::run() (loggerthread.cpp:8032) ==6516== by 0x804F7CE: LoggerThread::runStatic(void*) (loggerthread.cpp:9036) ==6516== by 0x1BB49DEB: start_thread (in /lib/tls/libpthread-0.60.so) ==6516== Address 0x1BD9F938 is 560 bytes inside a block of size 8192 free'd ==6516== at 0x1B903A5D: free (vg_replace_malloc.c:152) ==6516== by 0x1BABD505: apr_allocator_destroy (apr_pools.c:111) ==6516== by 0x1BABDB92: apr_pool_destroy (apr_pools.c:777) ==6516== by 0x1BABD783: apr_pool_terminate (apr_pools.c:568) ==6516== by 0x1BABECAA: apr_terminate (start.c:82) ==6516== by 0x1BA5CABB: log4cxx::helpers::APRInitializer::~APRInitializer() (aprinitializer.cpp:47) ==6516== by 0x1BA5CBEF: __tcf_0 (aprinitializer.cpp:53) ==6516== by 0x1BC63967: __cxa_finalize (in /lib/tls/libc-2.3.2.so) ==6516== by 0x1B9DFED8: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1BACFB19: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1B8F0EDC: _dl_fini (in /lib/ld-2.3.2.so) ==6516== by 0x1BC636D2: exit (in /lib/tls/libc-2.3.2.so) ==6516== ==6516== Invalid read of size 4 ==6516== at 0x1BB4B7A1: pthread_mutex_lock (in /lib/tls/libpthread-0.60.so) ==6516== by 0x1BABF1BE: apr_thread_mutex_lock (thread_mutex.c:92) ==6516== by 0x1BA75520: log4cxx::helpers::synchronized::synchronized(log4cxx::helpers::Mutex const&) (synchronized.cpp:30) ==6516== by 0x1BA7E7A6: log4cxx::AppenderSkeleton::doAppend(log4cxx::helpers::ObjectPtrT<log4cxx ::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderskeleton.cpp:93) ==6516== by 0x1BA980EF: log4cxx::helpers::AppenderAttachableImpl::appendLoopOnAppenders(log4cxx: :helpers::ObjectPtrT<log4cxx::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderattachableimpl.cpp:61) ==6516== by 0x1BA858BA: log4cxx::Logger::callAppenders(log4cxx::helpers::ObjectPtrT<log4cxx::spi ::LoggingEvent> const&, log4cxx::helpers::Pool&) (logger.cpp:78) ==6516== by 0x1BA85F3A: log4cxx::Logger::forcedLog(log4cxx::helpers::ObjectPtrT<log4cxx::Level> const&, std::string const&) (logger.cpp:129) ==6516== by 0x1BA880E2: log4cxx::Logger::debug(std::string const&) (logger.cpp:591) ==6516== by 0x8053057: LoggerThread::meth_0() (loggerthread.cpp:36) ==6516== by 0x804F904: LoggerThread::run() (loggerthread.cpp:8032) ==6516== by 0x804F7CE: LoggerThread::runStatic(void*) (loggerthread.cpp:9036) ==6516== by 0x1BB49DEB: start_thread (in /lib/tls/libpthread-0.60.so) ==6516== Address 0x1BD9F944 is 572 bytes inside a block of size 8192 free'd ==6516== at 0x1B903A5D: free (vg_replace_malloc.c:152) ==6516== by 0x1BABD505: apr_allocator_destroy (apr_pools.c:111) ==6516== by 0x1BABDB92: apr_pool_destroy (apr_pools.c:777) ==6516== by 0x1BABD783: apr_pool_terminate (apr_pools.c:568) ==6516== by 0x1BABECAA: apr_terminate (start.c:82) ==6516== by 0x1BA5CABB: log4cxx::helpers::APRInitializer::~APRInitializer() (aprinitializer.cpp:47) ==6516== by 0x1BA5CBEF: __tcf_0 (aprinitializer.cpp:53) ==6516== by 0x1BC63967: __cxa_finalize (in /lib/tls/libc-2.3.2.so) ==6516== by 0x1B9DFED8: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1BACFB19: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1B8F0EDC: _dl_fini (in /lib/ld-2.3.2.so) ==6516== by 0x1BC636D2: exit (in /lib/tls/libc-2.3.2.so) ==6516== ==6516== Invalid write of size 4 ==6516== at 0x1BB4B7A4: pthread_mutex_lock (in /lib/tls/libpthread-0.60.so) ==6516== by 0x1BABF1BE: apr_thread_mutex_lock (thread_mutex.c:92) ==6516== by 0x1BA75520: log4cxx::helpers::synchronized::synchronized(log4cxx::helpers::Mutex const&) (synchronized.cpp:30) ==6516== by 0x1BA7E7A6: log4cxx::AppenderSkeleton::doAppend(log4cxx::helpers::ObjectPtrT<log4cxx ::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderskeleton.cpp:93) ==6516== by 0x1BA980EF: log4cxx::helpers::AppenderAttachableImpl::appendLoopOnAppenders(log4cxx: :helpers::ObjectPtrT<log4cxx::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderattachableimpl.cpp:61) ==6516== by 0x1BA858BA: log4cxx::Logger::callAppenders(log4cxx::helpers::ObjectPtrT<log4cxx::spi ::LoggingEvent> const&, log4cxx::helpers::Pool&) (logger.cpp:78) ==6516== by 0x1BA85F3A: log4cxx::Logger::forcedLog(log4cxx::helpers::ObjectPtrT<log4cxx::Level> const&, std::string const&) (logger.cpp:129) ==6516== by 0x1BA880E2: log4cxx::Logger::debug(std::string const&) (logger.cpp:591) ==6516== by 0x8053057: LoggerThread::meth_0() (loggerthread.cpp:36) ==6516== by 0x804F904: LoggerThread::run() (loggerthread.cpp:8032) ==6516== by 0x804F7CE: LoggerThread::runStatic(void*) (loggerthread.cpp:9036) ==6516== by 0x1BB49DEB: start_thread (in /lib/tls/libpthread-0.60.so) ==6516== Address 0x1BD9F93C is 564 bytes inside a block of size 8192 free'd ==6516== at 0x1B903A5D: free (vg_replace_malloc.c:152) ==6516== by 0x1BABD505: apr_allocator_destroy (apr_pools.c:111) ==6516== by 0x1BABDB92: apr_pool_destroy (apr_pools.c:777) ==6516== by 0x1BABD783: apr_pool_terminate (apr_pools.c:568) ==6516== by 0x1BABECAA: apr_terminate (start.c:82) ==6516== by 0x1BA5CABB: log4cxx::helpers::APRInitializer::~APRInitializer() (aprinitializer.cpp:47) ==6516== by 0x1BA5CBEF: __tcf_0 (aprinitializer.cpp:53) ==6516== by 0x1BC63967: __cxa_finalize (in /lib/tls/libc-2.3.2.so) ==6516== by 0x1B9DFED8: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1BACFB19: (within /home/ewyles/log4cxx/log4cxx-0.9.8/build/debug/shared/liblog4cxx.so) ==6516== by 0x1B8F0EDC: _dl_fini (in /lib/ld-2.3.2.so) ==6516== by 0x1BC636D2: exit (in /lib/tls/libc-2.3.2.so) ==6516== ==6516== Invalid read of size 4 ==6516== at 0x1B9E1A4B: log4cxx::helpers::Transcoder::encode(std::string const&, std::string&) (transcoder.cpp:76) ==6516== by 0x1BA1A5C7: log4cxx::helpers::SystemOutWriter::write(std::string const&, log4cxx::helpers::Pool&) (systemoutwriter.cpp:52) ==6516== by 0x1BA1C6FB: log4cxx::WriterAppender::subAppend(log4cxx::helpers::ObjectPtrT<log4cxx: :spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (writerappender.cpp:199) ==6516== by 0x1BA1BCE4: log4cxx::WriterAppender::append(log4cxx::helpers::ObjectPtrT<log4cxx::sp i::LoggingEvent> const&, log4cxx::helpers::Pool&) (writerappender.cpp:81) ==6516== by 0x1BA7EA5D: log4cxx::AppenderSkeleton::doAppend(log4cxx::helpers::ObjectPtrT<log4cxx ::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderskeleton.cpp:125) ==6516== by 0x1BA980EF: log4cxx::helpers::AppenderAttachableImpl::appendLoopOnAppenders(log4cxx: :helpers::ObjectPtrT<log4cxx::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderattachableimpl.cpp:61) ==6516== by 0x1BA858BA: log4cxx::Logger::callAppenders(log4cxx::helpers::ObjectPtrT<log4cxx::spi ::LoggingEvent> const&, log4cxx::helpers::Pool&) (logger.cpp:78) ==6516== by 0x1BA85F3A: log4cxx::Logger::forcedLog(log4cxx::helpers::ObjectPtrT<log4cxx::Level> const&, std::string const&) (logger.cpp:129) ==6516== by 0x1BA880E2: log4cxx::Logger::debug(std::string const&) (logger.cpp:591) ==6516== by 0x8053057: LoggerThread::meth_0() (loggerthread.cpp:36) ==6516== by 0x804F904: LoggerThread::run() (loggerthread.cpp:8032) ==6516== by 0x804F7CE: LoggerThread::runStatic(void*) (loggerthread.cpp:9036) ==6516== Address 0x0 is not stack'd, malloc'd or (recently) free'd ==6516== ==6516== Process terminating with default action of signal 11 (SIGSEGV) ==6516== Access not within mapped region at address 0x0 ==6516== at 0x1B9E1A4B: log4cxx::helpers::Transcoder::encode(std::string const&, std::string&) (transcoder.cpp:76) ==6516== by 0x1BA1A5C7: log4cxx::helpers::SystemOutWriter::write(std::string const&, log4cxx::helpers::Pool&) (systemoutwriter.cpp:52) ==6516== by 0x1BA1C6FB: log4cxx::WriterAppender::subAppend(log4cxx::helpers::ObjectPtrT<log4cxx: :spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (writerappender.cpp:199) ==6516== by 0x1BA1BCE4: log4cxx::WriterAppender::append(log4cxx::helpers::ObjectPtrT<log4cxx::sp i::LoggingEvent> const&, log4cxx::helpers::Pool&) (writerappender.cpp:81) ==6516== by 0x1BA7EA5D: log4cxx::AppenderSkeleton::doAppend(log4cxx::helpers::ObjectPtrT<log4cxx ::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderskeleton.cpp:125) ==6516== by 0x1BA980EF: log4cxx::helpers::AppenderAttachableImpl::appendLoopOnAppenders(log4cxx: :helpers::ObjectPtrT<log4cxx::spi::LoggingEvent> const&, log4cxx::helpers::Pool&) (appenderattachableimpl.cpp:61) ==6516== by 0x1BA858BA: log4cxx::Logger::callAppenders(log4cxx::helpers::ObjectPtrT<log4cxx::spi ::LoggingEvent> const&, log4cxx::helpers::Pool&) (logger.cpp:78) ==6516== by 0x1BA85F3A: log4cxx::Logger::forcedLog(log4cxx::helpers::ObjectPtrT<log4cxx::Level> const&, std::string const&) (logger.cpp:129) ==6516== by 0x1BA880E2: log4cxx::Logger::debug(std::string const&) (logger.cpp:591) ==6516== by 0x8053057: LoggerThread::meth_0() (loggerthread.cpp:36) ==6516== by 0x804F904: LoggerThread::run() (loggerthread.cpp:8032) ==6516== by 0x804F7CE: LoggerThread::runStatic(void*) (loggerthread.cpp:9036) ==6516== ==6516== ERROR SUMMARY: 33 errors from 19 contexts (suppressed: 25 from 1) ==6516== malloc/free: in use at exit: 15572 bytes in 41 blocks. ==6516== malloc/free: 78 allocs, 37 frees, 92824 bytes allocated. ==6516== For counts of detected errors, rerun with: -v ==6516== searching for pointers to 41 not-freed blocks. ==6516== checked 805563604 bytes. ==6516== ==6516== LEAK SUMMARY: ==6516== definitely lost: 0 bytes in 0 blocks. ==6516== possibly lost: 4640 bytes in 8 blocks. ==6516== still reachable: 10932 bytes in 33 blocks. ==6516== suppressed: 0 bytes in 0 blocks. ==6516== Reachable blocks (those to which a pointer was found) are not shown. ==6516== To see them, rerun with: --show-reachable=yes Segmentation fault > -----Original Message----- > From: Curt Arnold [mailto:[EMAIL PROTECTED] > Sent: Tuesday, March 28, 2006 12:02 AM > To: Log4CXX User > Subject: Re: segmentation faults > > > On Mar 27, 2006, at 5:07 PM, Wyles Eric - ewyles wrote: > > > > > Curt, I was able to recreate this consistently using a test > > application > > and I was actually able to observe it through gdb and get a better > > stack > > trace (see below). Does this help? > > > > > > Program received signal SIGSEGV, Segmentation fault. > > [Switching to Thread -1342264400 (LWP 15890)] > > 0x002046c3 in log4cxx::helpers::ObjectPtrT<log4cxx::Logger>::operator= > > (this=0xc, [EMAIL PROTECTED]) at objectptr.h:101 > > 101 if (this->p != p.p) > > (gdb) bt > > #0 0x002046c3 in > > log4cxx::helpers::ObjectPtrT<log4cxx::Logger>::operator= (this=0xc, > > [EMAIL PROTECTED]) at objectptr.h:101 > > #1 0x002884f9 in log4cxx::Hierarchy::updateParents (this=0x9b1fbd0, > > [EMAIL PROTECTED]) at > > /home/ewyles/ags1.0code/log4cxx/src/hierarchy.cpp:327 > > > > ... > > If you could file a bug and attach the test application and describe > the platform (OS, compiler, processor type and number), it would be > very helpful. The stack trace looks unlike the previous one that > ended up in pthread, might not be the same problem but worth tracking > down. > > You previously mentioned that you were using a snapshot build. Was > this problem also observed with the current SVN HEAD? > > I don't see an obvious way to get to the state shown by the stack > trace, the newly created logger's parent member appears to be > corrupted or not yet initialized (this=0xc). There is a synchronized > constructor at line 168 in hierarchy.cpp that should have blocked > simultaneous modifications of the same logger, but maybe there is > another path that wasn't guarded. Are all the other threads in > innocuous locations at the time of the fault? > > Could you run the test under valgrind? Any diagnostics there? *************************************************************************** The information contained in this communication is confidential, is intended only for the use of the recipient named above, and may be legally privileged. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please resend this communication to the sender and delete the original message or any copy of it from your computer system. Thank You. ****************************************************************************
