I figured out the problem. The error was due to my system rebasing the libeay32.dll library from its desired base address of 0xFB00000. According to OpenSSL documents, this is supposed to generate a specific error message of FIPS_R_FINGERPRINT_DOES_NOT_MATCH_NONPIC_RELATED, but because I wasn't seeing that, I didn't think that was the problem.
However, process explorer showed that the base address of libeay32.dll in the tomcat7.exe process was not at its correct base address. I recompiled OpenSSL with a new base address, verified that the new dll wasn't being rebased, and then turned on FIPS mode, and it worked. With my test application, the original base address was not being changed by the OS, according to process explorer, which is why it worked with the original build. Thanks for your help! --Steve Nickels, Ipswitch, Inc. > -----Original Message----- > From: Christopher Schultz [mailto:ch...@christopherschultz.net] > Sent: Thursday, June 13, 2013 9:17 AM > To: Tomcat Users List > Subject: Re: TCNative with FIPS OpenSSL throws fingerprint error in FIPS > mode > > -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA256 > > Steve, > > On 6/12/13 6:54 PM, Steve Nickels wrote: > >>> I'm fairly confident that the OpenSSL library I'm using is valid and > >>> uncorrupted (I've used a couple different copies: an existing set of > >>> binaries being used successfully in another product internally, and > >>> a newly built version which I have successfully used the openssl > >>> utility against, without error). > >> > >> Can you write a simple C program to link against OpenSSL and try to > >> start it in FIPS mode? Does that work without error? Feel free to > >> just steal code from tcnative to put-together a Frankenstein's > >> monster of code just to see if it works. > > > > I've done so, and verified that my OpenSSL build seems to be working > > correctly, both in FIPS mode and not. My test program creates SHA-1 > > and MD5 hashes of a simple string value. With FIPS mode off, both > > hashes are returned. With FIPS mode on, the SHA-1 hash is returned, > > and the MD5 hash generates the expected "disabled for fips" error. > > There was no error at the point of FIPS_mode_set(1), which seems to > > indicate that the self tests passed. This matches what I saw when I > > used the openssl.exe utility that was compiled with OpenSSL (version > > OpenSSL 1.0.1c-fips 10 May 2012). > > > > Using this same OpenSSL build in tcnative, however, results in the > > fingerprint error when Tomcat starts up with FIPS mode enabled. > > > > > >>> My assumption is that I'm not building/linking OpenSSL correctly > >>> into tcnative. > >> > >> ...and you are building tcnative by hand because the OpenSSL Tomcat > >> provides is not build with FIPS compatibility, right? You will have > >> to make sure you have a FIPS-compatible OpenSSL (please post the > >> result of "openssl.exe version") and you will definitely have to > >> re-build tcnative against it because otherwise all the FIPS stuff > >> will generate errors before even trying to call FIPS_mode_set on > >> OpenSSL. > > > > Correct. I get the expected "FIPS not available" error when I turn on > > FIPS mode using the stock tcnative-1.dll library that comes with > > Tomcat. The FIPS-compatible OpenSSL build I have reports as "OpenSSL > > 1.0.1c-fips 10 May 2012". > > > > > >> I notice that Tomcat distributes openssl.exe and not openssl.dll (or > >> similar). Are you building openssl.exe or openssl.dll when you build > >> OpenSSL? > > > > Building OpenSSL on Windows results in three distributable files: > > libeay32.dll, ssleay32.dll, and openssl.exe. I copy the first two into > > Tomcat\bin, along with tcnative-1.dll, in order to make OpenSSL > > available to tcnative. It also results in libeay32.lib and > > ssleay32.lib, which are used in the tcnative compile process. > > What happens is you put openssl.exe in there alongside the .dll files? > > With your test program, was anything in the PATH (or current > directory) other than the two .dll files? (I'm just trying to figure out why > Tomcat ships with openssl.exe at all... I thought it was all > statically-linked). > > I presume you are not building a statically-linked tcnative.dll (which would > include the OpenSSL code), right? > > - -chris > -----BEGIN PGP SIGNATURE----- > Version: GnuPG/MacGPG2 v2.0.17 (Darwin) > Comment: GPGTools - http://gpgtools.org > Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ > > iQIcBAEBCAAGBQJRudRfAAoJEBzwKT+lPKRYeaAP/3DLdVl6BBLAAqKMANzQE > 4rv > uz44hZh6kpAyfE6ZKdmmm9WOw0X2g6Fq/rYDlYJlX0V/357AXQ33CttDXauU5 > eGs > N0gjbe9E75mIm7HJlXoKnK3U4HjU2/Pc16q1jCdcu8YW3NYVyztglMbOd/hjYc+ > Z > GRGk6Q+/qsI42As7GGMltLiO6FS+e4sgZ4fOlsppcn/w9g9GTCdENifKX1Dl851j > 8mqfNEVSrMJy1kxbXmVyvE/Nmv2eLsVZGyOAkqIMEeZFuloLuRBAK9o+EGJiL > 8Ff > /ewNsn1G2otNVi+8TyFFrOLZPV2MyveBzNw53umjYyVkmfxKIorm5LP2peeN > 0/53 > Zg6HrF1fV4LtAsqU/GWsa3j7O87kP4f1ZyYvYb+0BEMBeZtq/XNVWhzvADj9IV9 > s > x9QiPMVZTwZRaBO11p+mlRsam3tWWJpIGzxmUke9GHYOunKHKLed7S5ZGC > ZV5yje > 4jozK+x5ueNHb3rh/HJIJlo4534wFTxB1L9Xuq1/WT39lYydJ4wcF+51+BYDwAM > p > M1AnA37PBS76nXzOOu4nQo6LW9pSIQ59B57WpH8m3HRZvgO7xZxZWMs3V > dViC6A+ > OkCXqFORijvyG9YSbKun9D7NPIUZchynvEFQbmOE5K0A56Q9+u6v/76UuVItjiJ > q > biV1heB9VPjzArksryKq > =5w4s > -----END PGP SIGNATURE----- > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org For additional commands, e-mail: users-h...@tomcat.apache.org
