[openssl-dev] [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
many updates have been made, clsoing this ticket. re-open if needed. -- Rich Salz, OpenSSL dev team; rs...@openssl.org ___ openssl-dev mailing list To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-dev
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
Dr. Stephen Henson wrote: That's most peculiar. I've tested 1.0.0-stable with Visual Studio Express 2008. I did the following simple test: Start openssl.exe Start VS express. Select tools-attach to process. Debug-Break All. I get a warning about deadlocked process but it is otherwise OK. I can set breakpoints and source code is displayed. This worked for debug shared and static builds. The non-debug builds didn't show any source and it said there were no symbols. This is working here, too. But... I have some very different directories for OpenSSL source tree (d:\proj.svn\common\openssl-1.0.0), plus for my project code (d:\proj.svn\Training\ssl1\MrFtp) and finally for the installed OpenSSL binaries (aka '--prefix', here it is: C:\tools\OpenSSL). The crazy thing now comes (please kindly keep patience, the solution is also coming at end of the mail): I built OpenSSL 2 times. The first one, with additional /Zi goes to = C:\TOOLS\OpenSSL, The second one, with normal -Zi goes to = C:\temp\OpenSSL, I tried your debug/attach test - it works even for C:\temp\OpenSSL\bin\openssl.exe (no extra /Zi) Then I changed my local project to use -I C:\temp\OpenSSL\include and linker uses c:\temp\OpenSSL\lib. Expecting to _not_ beeing able to jump into ERR_load_BIO_strings(), I am surprised. It works now! Next, I changed back -I C:\TOOLS\OpenSSL\include and linker uses c:\TOOLS\OpenSSL\lib. This is, where I have my /Zi-extra-built installed. Surprisingly, now _this_ version is not able to be debugged anymore! Linker tells: 1Generating Code... 1Linking... 1libeay32.lib(cryptlib.obj) : warning LNK4099: PDB 'out32.pdb' was not found with 'c:\tools\openssl\lib\libeay32.lib' or at 'd:\proj.svn\Training\ssl1\debug\out32.pdb'; linking object as if no debug info 1libeay32.lib(mem.obj) : warning LNK4099: PDB 'out32.pdb' was not found with 'c:\tools\openssl\lib\libeay32.lib' or at 'd:\proj.svn\Training\ssl1\debug\out32.pdb'; linking object as if no debug info 1libeay32.lib(mem_dbg.obj) : warning LNK4099: PDB 'out32.pdb' was not found with 'c:\tools\openssl\lib\libeay32.lib' or at 'd:\proj.svn\Training\ssl1\debug\out32.pdb'; linking object as if no debug info The files out32.pdb / out32dll.pdb are still laying around in d:\proj.svn\common\openssl-1.0.0. Should this be the reason? wrong location of *.pdb files? I copied them into c:\TOOLS\OpenSSL\lib and my projects d:\proj.svn\Training\ssl1\debug folders. No help - linker is complaining they are not referring to the *right* library: 2Generating Code... 2Linking... 2libeay32.lib(cryptlib.obj) : warning LNK4204: 'd:\proj.svn\Training\ssl1\debug\out32.pdb' is missing debugging information for referencing module; linking object as if no debug info 2libeay32.lib(mem.obj) : warning LNK4204: 'd:\proj.svn\Training\ssl1\debug\out32.pdb' is missing debugging information for referencing module; linking object as if no debug info If I understand this right, without the extra /Zi, the _location_ and the _content_ of out32*.pdb files is very important. To give the final proof, I again built the whole OpenSSL with my default --prefix=c:\TOOLS\OpenSSL, but this time /without/ the extra /Zi. Then again recompiled and started my training project ... now the debugging again works. _Conclusion_: With standard build rules (configure, make, test, install) _without_ a patched '\util\pl\VC-32.pl', VS2005 debugging _is_ working for both process attach and Run/Debug-in-single-Steps. But only if some more prerequisites are kept in mind: * source directory tree (where INSTALL.W32 resides) shall not be cleaned or removed after successfully run of 'make install' * this source directory must contain out32[dll].pdb * out32[dll].pdb must be created together with the same make configure make install call, which created your [prefix]\include and [prefix]\lib directories and content * your C project needs to include exactly the mentioned [prefix]\include and [prefix]\lib components, because its path seems to be encoded in out32[dll].pdb and vice versa. _Conclusion_ II: With standard build rules (configure, make, test, install) _plus_ a patched '\util\pl\VC-32.pl' (added /Zi), VS2005 debugging _is_ working for both process attach and Run/Debug-in-single-Steps, without some of the prerequisites mentioned above. The PDB files are neither in the openssl directory nor in the [prefix] directory. But debuging works. puh, long tests, long mail, short conclusions. Hope this helps for somebody, Modem Man __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
On Thu, Apr 29, 2010, Modem Man wrote: wait a minute... ms\nt.mak (and compareable ntdll.mak) with original .pl file: CFLAG= /MTd /Od -DDEBUG -D_DEBUG -DOPENSSL_THREADS -DDSO_WIN32 -W3 -WX -Gs0 -GF -Gy -Zi -nologo -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE /Fdout32 -DOPENSSL_NO_RC5 -DOPENSSL_NO_MD2 -DOPENSSL_NO_KRB5 -DOPENSSL_NO_JPAKE -DOPENSSL_NO_DYNAMIC_ENGINE with /Zi enriched .pl file: CFLAG= /MTd /Zi /Od -DDEBUG -D_DEBUG -DOPENSSL_THREADS -DDSO_WIN32 -W3 -WX -Gs0 -GF -Gy -Zi -nologo -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE /Fdout32 -DOPENSSL_NO_RC5 -DOPENSSL_NO_MD2 -DOPENSSL_NO_KRB5 -DOPENSSL_NO_JPAKE -DOPENSSL_NO_DYNAMIC_ENGINE The only difference is just the additional /Zi, plus the fact that this version works for IDE debugging. Hmm, it looks like -Zi doesn't work at all or doesn't work at that particular place? Crazy... What shall I test next? That's most peculiar. I've tested 1.0.0-stable with Visual Studio Express 2008. I did the following simple test: Start openssl.exe Start VS express. Select tools-attach to process. Debug-Break All. I get a warning about deadlocked process but it is otherwise OK. I can set breakpoints and source code is displayed. This worked for debug shared and static builds. The non-debug builds didn't show any source and it said there were no symbols. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
Dr. Stephen Henson schrieb: On Wed, Apr 28, 2010, Modem Man wrote Stephen Henson via RT schrieb: [mounir.idra...@idrix.net - Mon Apr 26 20:18:42 2010]: Hi, This patch adds the /Zi switch to CFLAG in the debug configuration in order to permit stepping inside OpenSSL code during debug sessions. It applied to the latest snapshots of 1.0.0 and 0.9.8 source trees. Note that 1.0.0 makes the Win32 build system a bit saner: it uses some command line options from the Configure line (like other targets). The debug-VC-WIN32 target already includes -Zi (same as /Zi) so that change is unnecessary. Dear Stephen, I'm using VS2005 and built my 1.0.0 with 'debug-VC-WIN32', and there was no /Zi at all. Mounir gave me the hint and I modified my .\util\pl\VC-32.pl very similiar to his patch. AFTER doing this, it worked for me. BEFORE doing this, I was not able to jump into the code. Not even into the x86 asm representation, which is shown by VS in cases, the source can't be found. So in my opinion, it is indeed required patch for 1.0.0 With great respect about your work, Modem Man That's strange. I've just tried this in the latest 1.0.0-stable tree: perl Configure debug-VC-WIN32 ms\do_nasm and in ms\ntdll.mak is this: CFLAG= /MDd /Od -DDEBUG -D_DEBUG -DOPENSSL_THREADS -DDSO_WIN32 -W3 -Gs0 -GF -Gy -Zi -nologo ... Note the -Zi and not /Zi. Does that not happen on your system? wait a minute... ms\nt.mak (and compareable ntdll.mak) with original .pl file: CFLAG= /MTd /Od -DDEBUG -D_DEBUG -DOPENSSL_THREADS -DDSO_WIN32 -W3 -WX -Gs0 -GF -Gy -Zi -nologo -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE /Fdout32 -DOPENSSL_NO_RC5 -DOPENSSL_NO_MD2 -DOPENSSL_NO_KRB5 -DOPENSSL_NO_JPAKE -DOPENSSL_NO_DYNAMIC_ENGINE with /Zi enriched .pl file: CFLAG= /MTd /Zi /Od -DDEBUG -D_DEBUG -DOPENSSL_THREADS -DDSO_WIN32 -W3 -WX -Gs0 -GF -Gy -Zi -nologo -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -D_CRT_SECURE_NO_DEPRECATE /Fdout32 -DOPENSSL_NO_RC5 -DOPENSSL_NO_MD2 -DOPENSSL_NO_KRB5 -DOPENSSL_NO_JPAKE -DOPENSSL_NO_DYNAMIC_ENGINE The only difference is just the additional /Zi, plus the fact that this version works for IDE debugging. Hmm, it looks like -Zi doesn't work at all or doesn't work at that particular place? Crazy... What shall I test next? with best regards, Modem Man
[openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
[mounir.idra...@idrix.net - Mon Apr 26 20:18:42 2010]: Hi, This patch adds the /Zi switch to CFLAG in the debug configuration in order to permit stepping inside OpenSSL code during debug sessions. It applied to the latest snapshots of 1.0.0 and 0.9.8 source trees. Note that 1.0.0 makes the Win32 build system a bit saner: it uses some command line options from the Configure line (like other targets). The debug-VC-WIN32 target already includes -Zi (same as /Zi) so that change is unnecessary. I'll update 0.9.8. Note that it is also possible to add -Zi on the command line to Configure for non-debug builds to avoid needing to modify OpenSSL of the generated Makefile. Is anyone aware of any side effects or incompatibilities caused by always using -Zi? At present there is no option to remove a default option so that would require manual changes to the generated makefile under Win32. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
Stephen Henson via RT schrieb: [mounir.idra...@idrix.net - Mon Apr 26 20:18:42 2010]: Hi, This patch adds the /Zi switch to CFLAG in the debug configuration in order to permit stepping inside OpenSSL code during debug sessions. It applied to the latest snapshots of 1.0.0 and 0.9.8 source trees. Note that 1.0.0 makes the Win32 build system a bit saner: it uses some command line options from the Configure line (like other targets). The debug-VC-WIN32 target already includes -Zi (same as /Zi) so that change is unnecessary. Dear Stephen, I'm using VS2005 and built my 1.0.0 with 'debug-VC-WIN32', and there was no /Zi at all. Mounir gave me the hint and I modified my .\util\pl\VC-32.pl very similiar to his patch. AFTER doing this, it worked for me. BEFORE doing this, I was not able to jump into the code. Not even into the x86 asm representation, which is shown by VS in cases, the source can't be found. So in my opinion, it is indeed required patch for 1.0.0 With great respect about your work, Modem Man
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
On Wed, Apr 28, 2010, Modem Man wrote: Stephen Henson via RT schrieb: [mounir.idra...@idrix.net - Mon Apr 26 20:18:42 2010]: Hi, This patch adds the /Zi switch to CFLAG in the debug configuration in order to permit stepping inside OpenSSL code during debug sessions. It applied to the latest snapshots of 1.0.0 and 0.9.8 source trees. Note that 1.0.0 makes the Win32 build system a bit saner: it uses some command line options from the Configure line (like other targets). The debug-VC-WIN32 target already includes -Zi (same as /Zi) so that change is unnecessary. Dear Stephen, I'm using VS2005 and built my 1.0.0 with 'debug-VC-WIN32', and there was no /Zi at all. Mounir gave me the hint and I modified my .\util\pl\VC-32.pl very similiar to his patch. AFTER doing this, it worked for me. BEFORE doing this, I was not able to jump into the code. Not even into the x86 asm representation, which is shown by VS in cases, the source can't be found. So in my opinion, it is indeed required patch for 1.0.0 With great respect about your work, Modem Man That's strange. I've just tried this in the latest 1.0.0-stable tree: perl Configure debug-VC-WIN32 ms\do_nasm and in ms\ntdll.mak is this: CFLAG= /MDd /Od -DDEBUG -D_DEBUG -DOPENSSL_THREADS -DDSO_WIN32 -W3 -Gs0 -GF -Gy -Zi -nologo ... Note the -Zi and not /Zi. Does that not happen on your system? Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
On Thu, Apr 29, 2010 at 12:31 AM, Ger Hobbelt g...@hobbelt.com wrote: And for those that like this to be like this to be -- liking this to :-( On Thu, Apr 29, 2010 at 12:31 AM, Ger Hobbelt g...@hobbelt.com wrote: And for those that like this to be like this to be -- liking this to:-(
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
Steve, In answer to your question: On Wed, Apr 28, 2010 at 3:19 PM, Stephen Henson via RT r...@openssl.orgwrote: Note that it is also possible to add -Zi on the command line to Configure for non-debug builds to avoid needing to modify OpenSSL of the generated Makefile. Is anyone aware of any side effects or incompatibilities caused by always using -Zi? At present there is no option to remove a default option so that would require manual changes to the generated makefile under Win32. I would advise not to have -Zi in /all/ builds for the following reasons: - The /Zi option has modest to severe impact on the way the linker works. See below for references; this is about the /OPT implied by the use of /Zi, which disables a few .exe/.dll space optimizations in the linker unless specific action is taken (i.e. explicit /OPT overrides on the command line. When not spending that extra effort, the final result is /at least/ a (slightly?) different memory footprint of the generated .dll/.exe (.exe in case of a static lib linked into app code). (And for those that like this to be triviality galore, there's the /LTCG bits as well. See also the /LTCG bit below.) - AFAICT the generated PE files will have at least a marker in their header to make debuggers aware that a separate .PDB debug database exists. In commercial settings it is often frowned upon to deliver binaries which have [the risk of] carrying any sort of debug info in them. (Sure, the debug info ain't /in/ there, but you have a high risk more symbols end up in your DLL/EXE than you might like. For those not yet in the know, there's a /very/ nice disassembler called IDA which can do very nice things for you when you need it. Any little bit of debuggy data bits help, there.) Bottom line: IMHO it's easier to *add* a /Zi (or -Zi) for an explicit developer-targetting 'debug build' than to go the several extra miles (and run the risk of a screwup some time later on in the maintenance cycle of the software lifetime) in trying to *remove* any undesirable 'side effect' of /Zi in any production-targetting 'release build'. Enough bla bla. References. Here they are: http://msdn.microsoft.com/en-us/library/958x11bc%28VS.80%29.aspx The official word of Microsoft on this one. :-) No ill effect on the code optimizer here. Note the mention of 'implied /DEBUG', so that one's next: http://msdn.microsoft.com/en-us/library/xe4t6fc1%28v=VS.80%29.aspx which happens to 'imply' non-default /OPT behaviour (read: non-default for 'release builds'!), so that's next: (Oh, and please do notice /INCREMENTAL being mentioned! You'll see the relevance when we get to /LTCG...) http://msdn.microsoft.com/en-us/library/bxwfs976%28v=VS.80%29.aspx which describes what the two effected options /OPT:REF|NOREF and /OPT:ICF|NOICF do exactly. They both impact the generated binary at link time. /OPT:REF: LINK removes unreferenced packaged functions i.e. different footprint and 'hence' link-time whole-program optimizations will have to produce different results. Also note that sneaky single line where it says you can do whatever you like but you won't EVER get your 'default' == regular linker behaviour back again: Note that there is a difference in linker behavior when *ICF* is in effect by default [...]. See more at the /OPT:ICF section about this. Re 'whole program optimization' (what it's called in the MSVC IDE's): that's the /LTCG (Link-time Code Generation) linker option: http://msdn.microsoft.com/en-US/library/xbf3tbeh%28v=VS.80%29.aspx which can do great things for your binary :-) ... unless you hit the snag mentioned in just one itty bitty line in the man page right there: /LTCG is not valid for use with /INCREMENTALhttp://msdn.microsoft.com/en-US/library/4khtbfyf%28v=VS.80%29.aspx. ... Oops. Bye bye, /LTCG. Twas knowing you. Re .PDB format and related materials, there's the mentioned (in the /DEBUG manpage) KnowledgeBase article Q121366: http://support.microsoft.com/kb/121366 which (my interpretation) is a lot of text with little relevance to this item, except maybe the note about It is possible to strip debug information from a PE file [...]. And in closing, since I can't vouch for the makefiles any which way as I haven't used them for at least a decade (always used our in-house dedicated Visual Studio project files instead), here's two bits of discussion where people talk about Windows debug vs. release builds compared to Unix, pardon, Linux practices with respect to 'stripping debug info' like operations. FYI: http://priyeshwadhwa.blogspot.com/ http://www.eggheadcafe.com/software/aspnet/30629196/strip-debugging-informati.aspx Bottom line (my paraphrasing): Microsofties don't /strip/ debug info from binaries; they have a more explicit debug versus release build approach. It probably explains why I always like to have a ./configure --debug-build similar and then get to find that I haven't, most of the time. (This is generally speaking, not about OpenSSL)
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
Hi, I have on purpose only added /Zi to the debug build because it is not always desirable to add symboles to release builds whereas it is always needed for debug ones. -- Mounir IDRASSI IDRIX http://www.idrix.fr On 4/26/2010 11:58 PM, William A. Rowe Jr. wrote: On 4/26/2010 1:18 PM, Mounir IDRASSI via RT wrote: Hi, This patch adds the /Zi switch to CFLAG in the debug configuration in order to permit stepping inside OpenSSL code during debug sessions. It applied to the latest snapshots of 1.0.0 and 0.9.8 source trees. It should be in base_cflags, since it is required to produce something from a crash dump that can be analyzed. Apparently half of a patch was applied without thought to this, you are certainly right that in the current state, the win32 build results are worthless to someone creating a release and to someone trying to debug the build. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
On 4/27/2010 5:35 AM, Mounir IDRASSI wrote: Hi, I have on purpose only added /Zi to the debug build because it is not always desirable to add symboles to release builds whereas it is always needed for debug ones. No, it's always desirable, and actually irresponsible not to track symbols during a build. It might not always be desirable to *distribute* those symbols. Different topic. Building with /Zi, linking /debug /opt:noref /incremental:no results in the same binary as if /Zi was omitted, and in a seperate symbols .pdb that may be discarded (foolish), retained by the packager for later forensics, or distributed with the binaries. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
On 4/27/2010 5:35 AM, Mounir IDRASSI wrote: Hi, I have on purpose only added /Zi to the debug build because it is not always desirable to add symboles to release builds whereas it is always needed for debug ones. No, it's always desirable, and actually irresponsible not to track symbols during a build. It might not always be desirable to *distribute* those symbols. Different topic. Building with /Zi, linking /debug /opt:noref /incremental:no results in the same binary as if /Zi was omitted, and in a seperate symbols .pdb that may be discarded (foolish), retained by the packager for later forensics, or distributed with the binaries. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
On 4/26/2010 1:18 PM, Mounir IDRASSI via RT wrote: Hi, This patch adds the /Zi switch to CFLAG in the debug configuration in order to permit stepping inside OpenSSL code during debug sessions. It applied to the latest snapshots of 1.0.0 and 0.9.8 source trees. It should be in base_cflags, since it is required to produce something from a crash dump that can be analyzed. Apparently half of a patch was applied without thought to this, you are certainly right that in the current state, the win32 build results are worthless to someone creating a release and to someone trying to debug the build. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
Re: [openssl.org #2245] [PATCH] Add /Zi to VC++ CFLAG in debug configuration (1.0.0 and 0.9.8)
On 4/26/2010 1:18 PM, Mounir IDRASSI via RT wrote: Hi, This patch adds the /Zi switch to CFLAG in the debug configuration in order to permit stepping inside OpenSSL code during debug sessions. It applied to the latest snapshots of 1.0.0 and 0.9.8 source trees. It should be in base_cflags, since it is required to produce something from a crash dump that can be analyzed. Apparently half of a patch was applied without thought to this, you are certainly right that in the current state, the win32 build results are worthless to someone creating a release and to someone trying to debug the build. __ OpenSSL Project http://www.openssl.org Development Mailing List openssl-dev@openssl.org Automated List Manager majord...@openssl.org
