Re: linking release cryptoPP dll to debug application results in heap corruption

[Voronkov Konstantin]

Hello, All!   We made analysis and found the problem reason: The problem happen because cryptoopp is trying to allocate memory using debug  version of runtime library, but the last one is still not initialized. This is because cryptopp using hack during initialization process, by searching for debug CRT (C Runtime) in process address space and if found getting addresses of global new/delete operators (if debug CRT not found cryptoPP is getting addresses from release one) for future memory allocation. Then after address of new and delete operators were resolved they are used during global variable initialization. So we have follwing situation: 1. debug.exe starting 2. OS is loading all DLLs dependance tree debug.exe depends on. 3. releaseCryptoPP is loading and resolving addresses of new/delete from debugCRT (releaseCRT already initialized, but debug one not) 4. cryptoPP using resolved addresses to allocate memory 5. we have access violation 6. here debugCRT will only initialize Note: debugCRT *should* be initialized before its first usage in application, so compiler absolutely sure that debugCRT is still not used, but cryptoPP is cheating by resolving addresses manually. If there would be at least one debugDLL loaded before cryptoPP then debugCRT would be initialized already... ================================ One more analysis (the same): The reason of the problem: - the msvcrtd.dll is not initialized in CryptoPP, but loaded on CryptoPP   initialization when new operator is called from msvcrtd.dll at this moment - this (loaded, but not initialized msvcrtd.dll) happens on some    conditions - depends on the order of dll initialization,   that is why problem is not easy to reproduce - the new is called, but because msvcrtd.dll is not   initialized it tries to read some non-initialized memory ================================ compiler options for application: /nologo /MDd /W3 /GR /GX /ZI /Od /D "_DEBUG" /D "WIN32" /D "_WINDOWS" /D "_AFXDLL" /FR"debug/" /Fp"debug/Messanger.pch" /Yu"stdafx.h" /Fo"debug/" /Fd"debug/" /FD /GZ /Zm200 /c here is the stack: NTDLL! 77f92373() NTDLL! 77f922f8() _nh_malloc_dbg(unsigned int 56, int 1, int 1, const char * 0x00000000, int 0) line 242 + 7 bytes _nh_malloc(unsigned int 56, int 1) line 197 + 19 bytes operator new(unsigned int 56) line 24 + 11 bytes CRYPTOPP! 4292a519() CRYPTOPP! 429492cc() NTDLL! 77f86215() NTDLL! 77f86f17() NTDLL! 77f8b845() NTDLL! 77f8c295() NTDLL! 77fa15d3()   We work around the problem by:  - removed from CryptoPP new and delete finding mechanism and linked all our  applications and dlls to release run-time library (even debug configuration).   Best Regards, Voronkov Konstantin   ----- Original Message ----- From: Voronkov Konstantin To: [EMAIL PROTECTED] Sent: Tuesday, November 02, 2004 3:23 PM Subject: linking release cryptoPP dll to debug application results in heap corruption Hello, All!   I want to ask you help about heap corruption with CryptoPP dll,  when I try to link release CryptoPP dll to debug version of applicaton. In addition I compiled CryptoPP and mine application with STLPort library.   I have problem linking release cryptoPP dll to debug  application, the cryptoPP gives access violation on exit.   I exported GetNewAndDeleteForCryptoPP to override new  and delete in cryptoPP and see that problem happens on memory freeing. The stack looks like:   NTDLL! 7c901230() NTDLL! 7c96cd80() NTDLL! 7c960af8() KERNEL32! 7c85e7af() _CrtIsValidHeapPointer(const void * 0x00897918) line 1697 _free_dbg_lk(void * 0x00897918, int 1) line 1044 + 9 bytes _free_dbg(void * 0x00897918, int 1) line 1001 + 13 bytes free(void * 0x00897918) line 956 + 11 bytes CRYPTOPP! 4293c49b() 72432073()   I'm sure that new and delete is overrided and called from CryptoPP,  because in case I set breakpoints inside new and delete I have calls from CryptoPP of mine overrided new and delete operators.   Maybe it will help you, but I noticed interesting fact, that the content  of data to free (passed to delete function) is readable and is: ThisObject:class CryptoPP::DL_GroupParameters_IntegerBasedImpl  <class CryptoPP::ModExpPrecomputation, class CryptoPP::DL_FixedBasePrecomputationImpl <class CryptoPP::Integer> >.««««««««............„...ThisObject:class  CryptoPP::DL_GroupParameters_IntegerBasedImpl<class CryptoPP::ModExpPrecomputation, class CryptoPP::DL_FixedBasePrecomputationImpl<class CryptoPP::Integer> >.««««««««   Best Regards, Voronkov Konstantin