You need to link to the static C libraries and not the dynamic ones. Memoverride.cpp causes the link to use our version of malloc(), free(), etc when using the static library (and these calls all get directed into the tier0 memory management code).
- Alfred -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Jorge Rodriguez Sent: Saturday, October 01, 2005 2:57 PM To: [email protected] Subject: [hlcoders] memoverride.cpp voodoo! It's been a while since I've been on this list... since November I think. I have a real stumper for my first question. This is not for the faint of heart, but it is a challenging problem that needs to be solved in the long run (and I hope has already been solved by someone that I haven't found yet!) It's fairly involved, and requires a good understanding (that I don't have!) of how the Visual C++ standard libraries work. Also, for the record, I would *highly* appreciate it if a *Valve employee* who is knowlegable about how memoverride.cpp works would read it and *provide their thoughts*. :D Thanks! First the backstory: I have ported the HL2SDK to CMake. For those of you who don't know, CMake is a build system that builds compiler-specific makefiles. For example, you can tell CMake to make you Visual Studio project files for any Visual Studio version, or NMake makefiles, or whatever else you please. This makes supporting multiple platforms and compiler versions a moot point, so long as you adhere to ANSI-C and use cross-platform libraries. Other strengths of CMake include out-of-source builds and automated building and testing. So anyways, I got all of the HL2SDK to build (the from scratch sdk version, not the hl2 mp or sp version) inside CMake, except for memoverride.cpp, which produces these warnings when it is built: memoverride.cpp \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(107) : warning C4273: 'malloc' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(112) : warning C4273: 'free' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(117) : warning C4273: 'realloc' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(122) : warning C4273: 'calloc' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(153) : warning C4273: '_heapchk' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(158) : warning C4273: '_heapmin' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(163) : warning C4273: '_msize' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(183) : warning C4273: '_expand' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(190) : warning C4273: '_get_sbh_threshold' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(195) : warning C4273: '_set_sbh_threshold' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(200) : warning C4273: '_heapadd' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(205) : warning C4273: '_heapset' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(210) : warning C4273: '_heapused' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(216) : warning C4273: '_heapwalk' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(358) : warning C4273: '_malloc_dbg' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(365) : warning C4273: '_calloc_dbg' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(374) : warning C4273: '_realloc_dbg' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(381) : warning C4273: '_expand_dbg' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(387) : warning C4273: '_free_dbg' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(393) : warning C4273: '_msize_dbg' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(445) : warning C4273: '_CrtDumpMemoryLeaks' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(450) : warning C4273: '_CrtSetDumpClient' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(455) : warning C4273: '_CrtSetDbgFlag' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(459) : warning C4273: '_crtBreakAlloc' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(460) : warning C4273: '_crtDbgFlag' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(468) : warning C4273: '_CrtSetAllocHook' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(474) : warning C4273: '_CrtSetBreakAlloc' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(479) : warning C4273: '_CrtIsValidHeapPointer' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(484) : warning C4273: '_CrtIsValidPointer' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(489) : warning C4273: '_CrtCheckMemory' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(496) : warning C4273: '_CrtIsMemoryBlock' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(502) : warning C4273: '_CrtMemDifference' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(508) : warning C4273: '_CrtMemDumpStatistics' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(513) : warning C4273: '_CrtMemCheckpoint' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(519) : warning C4273: '_CrtMemDumpAllObjectsSince' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(524) : warning C4273: '_CrtDoForAllClientObjects' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(532) : warning C4273: '_crtAssertBusy' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(535) : warning C4273: '_CrtSetReportMode' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(540) : warning C4273: '_CrtSetReportFile' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(545) : warning C4273: '_CrtSetReportHook' : inconsistent dll linkage \svn\hlc-dev\trunk\src\public\tier0\memoverride.cpp(551) : warning C4273: '_CrtDbgReport' : inconsistent dll linkage Information on this warning: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vccore/ html/C4273.asp (No help at all) Despite the warnings, I forge ahead bravely! The project would not link at all due to duplicate symbols: Linking... MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: __free_dbg already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: __malloc_dbg already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: __realloc_dbg already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: _malloc already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: _free already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: __msize already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: __msize_dbg already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : error LNK2005: __heapchk already defined in memoverride.obj MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __free_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __malloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __realloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _malloc already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _free already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __msize already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __msize_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __heapchk already defined in memoverride.obj; second definition ignored Creating library C:\build\hl2sdk\lib\Debug\client.lib and object C:\build\hl2sdk\lib\Debug\client.exp C:\build\hl2sdk\lib\Debug\client.dll : fatal error LNK1169: one or more multiply defined symbols found After a bit of googling, I discovered that someone porting the HL2SDK had solved this problem using the /FORCE:MULTIPLE flag to link. Using that produced these warnings while linking: Linking... LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/FORCE' specification MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __free_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __malloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __realloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _malloc already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _free already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __msize already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __msize_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __heapchk already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __free_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __malloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __realloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _malloc already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _free already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __msize already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __msize_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __heapchk already defined in memoverride.obj; second definition ignored Creating library C:\build\hl2sdk\lib\Debug\client.lib and object C:\build\hl2sdk\lib\Debug\client.exp C:\build\hl2sdk\lib\Debug\client.dll : warning LNK4088: image being generated due to /FORCE option; image may not run and Linking... LINK : warning LNK4075: ignoring '/INCREMENTAL' due to '/FORCE' specification MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __malloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __free_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __realloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _malloc already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _free already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __heapchk already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __malloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __free_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __realloc_dbg already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _malloc already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: _free already defined in memoverride.obj; second definition ignored MSVCRTD.lib(MSVCR71D.dll) : warning LNK4006: __heapchk already defined in memoverride.obj; second definition ignored Creating library C:\build\hl2sdk\lib\Debug\server.lib and object C:\build\hl2sdk\lib\Debug\server.exp C:\build\hl2sdk\lib\Debug\server.dll : warning LNK4088: image being generated due to /FORCE option; image may not run So, despite the warnings, I charge ahead at full speed! After smoothing out some problems with the port to CMake, I was successful in getting the client.dll and server.dll I made to run, boot, and load sdk_vehicles, however as soon as the MOTD closes, the game crashes. Here's the stack trace: NTDLL.DLL!7c901230() NTDLL.DLL!7c96c943() NTDLL.DLL!7c96cd80() NTDLL.DLL!7c960af8() KERNEL32.DLL!7c85e7af() > msvcr71d.dll!_CrtIsValidHeapPointer(const void * pUserData=0x028fd0c4) Line 1807 C msvcr71d.dll!_free_dbg_lk(void * pUserData=0x028fd0c4, int nBlockUse=1) Line 1132 + 0x9 C msvcr71d.dll!_free_dbg(void * pUserData=0x028fd0c4, int nBlockUse=1) Line 1070 + 0xd C client.dll!CUtlMemory<Vector>::Purge() Line 383 + 0x10 C++ client.dll!CUtlVector<Vector,CUtlMemory<Vector> >::Purge() Line 615 C++ client.dll!CUtlVector<Vector,CUtlMemory<Vector> >::~CUtlVector<Vector,CUtlMemory<Vector> >() Line 194 C++ client.dll!EmitSound_t::~EmitSound_t() + 0x19 C++ client.dll!C_BaseEntity::EmitSound(const char * soundname=0x216a5db4, float soundtime=0.00000000, float * duration=0x00000000) Line 996 + 0xf C++ client.dll!C_BaseCombatWeapon::OnDataChanged(DataUpdateType_t updateType=DATA_UPDATE_DATATABLE_CHANGED) Line 126 C++ client.dll!ProcessOnDataChangedEvents() Line 1127 + 0x14 C++ client.dll!OnRenderStart() Line 1202 C++ client.dll!CHLClient::FrameStageNotify(ClientFrameStage_t curStage=FRAME_RENDER_START) Line 1259 C++ engine.dll!0da634e7() engine.dll!0daef3bf() engine.dll!0daf9edc() engine.dll!0dafa573() engine.dll!0db04f75() engine.dll!0db05062() engine.dll!0db050ff() engine.dll!0dba2c4c() engine.dll!0dba2817() MaterialSystem.dll!0d51d0c5() engine.dll!0db24ceb() engine.dll!0dba2a9e() engine.dll!0dba28d4() engine.dll!0dba28df() engine.dll!0dba297c() launcher.dll!1000369b() launcher.dll!10007ff0() launcher.dll!10007ff0() launcher.dll!100057f8() You can immediately see the problem here: the call to free() has resolved to the one in msvcrtd.lib instead of the one in memoverride.obj. But the linker said, "__free_dbg already defined in memoverride.obj; second definition ignored" ! That states directly to me that the compiler kept the version in memoverride.obj and threw out the version in msvcrt.lib, but in fact, just the opposite apparently has happened. To be more specific as to what happened to cause this crash, in the file SoundEmitterSystem.cpp on line 989 there is an EmitSound_t which is created. At the end of the function, this object is destroyed, and a member of that structure (a CUtlVector) is cleaned up. That vector uses CUtlMemory to manage memory for itself, which uses malloc/free to allocate and free memory. Since the vector class is tier1 code, the engine has its own copy, which uses memoverride.cpp's malloc/free, which allocates memory on its own heap. Then along comes client.dll which uses the stdlib free to attempt to free that memory, and since that memory was not allocated on the stdlib heap, stdlib produces an earth-shattering kaboom worthy of Marvin the Marshan (or in other words fails an assert.) Upon further investigation, any call to free, malloc, calloc, realloc, or any other of the memoverride.cpp functions is not handled by memoverride.obj but by msvcrtd.lib instead. However, upon even further investigation, the new and delete operators work exactly as they should, and you will notice that they are not listed in any of the above warnings. Also, since the check in controls.cpp for _heapmin works correctly, only some of the memoverride.cpp are actually overriding msvcrtd.lib, and others are not. So, basically, memoverride.cpp isn't working as it should. So at last the question is: What kind of voodoo magic did Valve use to get this to work properly, and without any errors or even warnings? I'm astonished that in the default Valve SDK, memoverride.cpp doesn't even generate any warnings! If you would like to try it out for yourself, you can find the SDK here: http://www.vinoisnotouzo.com/hl2sdk-cmake.rar You'll need to hop on over to http://www.cmake.org to pick up a copy of CMake, I am using CMake 2.0, but 2.2 is available if you feel like using that instead. Uncompress the SDK, and run CMake on it. You can use the gui version, or run "cmake \hl2sdk-cmake\" from the build directory. Make sure you specify the build directory to be outside your source code directory. I chose \svn\hlc-sdk\trunk\src for my source directory and \build\hl2sdk for my build directory, yours can be whatever. Open the Visual Studio project file that CMake creates. Compile and debug. Included are the default SDK VS 7.1 project files for comparison. If you need assistance, send me an off-list email, or catch me on irc.gamesurge.net by the nick Vino, and I'll help you out. Lastly, I know that at least one person will ask this question: "Why don't you just use the default SDK project files instead of having to be a rebel all the time? You're digging yourself into a hole!" the answer to that question is, eventually people will need to know the answer to this question. If you don't think I should be doing this, then you don't have to help me. I think knowing the answer is worth the time it takes to figure this out. Phew! OK I'm done. -- Jorge "Vino" Rodriguez _______________________________________________ To unsubscribe, edit your list preferences, or view the list archives, please visit: http://list.valvesoftware.com/mailman/listinfo/hlcoders _______________________________________________ To unsubscribe, edit your list preferences, or view the list archives, please visit: http://list.valvesoftware.com/mailman/listinfo/hlcoders
