Environment: Windows 7 (I know, I know - I just hate Windows 10). Compiler: Visual Studio, have tried both VS2008 Pro and VS2019 Pro OpenSSL Build: 1.1.1g, retrieved from OpenSSL.org last night
I've been attempting to build OpenSSL 1.1.x since it came out, but each time I do so, I find that, while it compiles and links cleanly, it fails about 50% of its self tests when I perform "nmake test". It has been this way for several releases. By "fail" I mean that there's a stream of "dubious..." outputs that look like this excerpt: -------------------------- Cut here ---------------------------- ... test\recipes\03-test_internal_siphash.t ......... ok test\recipes\03-test_internal_sm2.t ............. ok test\recipes\03-test_internal_sm4.t ............. ok test\recipes\03-test_internal_ssl_cert_table.t .. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests test\recipes\03-test_internal_x509.t ............ ok test\recipes\03-test_ui.t ....................... ok test\recipes\04-test_asn1_decode.t .............. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests test\recipes\04-test_asn1_encode.t .............. Dubious, test returned 1 (wstat 256, 0x100) Failed 1/1 subtests ... -------------------------- Cut here ---------------------------- Each time I went through the process, I saw the long string of self-test failures and decided I'd put off migrating to 1.1.1 until it was sorted out, but it's been the same for at least four releases now. I finally decided I needed to track down what was going on, so I extrapolated how to run the failing tests manually with more verbose output from the OpenSSL wiki pages (which are just a little out of date). It appears that for at least the first twenty or thirty of these failures, the reason is because the test application has been compiled without including the required Applink code - a verbose output typically looks like this: -------------------------- Cut here ---------------------------- O:\ >perl test\recipes\05-test_idea.t 1..1 OPENSSL_Uplink(5C790330,08): no OPENSSL_Applink ..\ideatest.exe => 1 not ok 1 - running ideatest # Failed test 'running ideatest' # at util/perl/OpenSSL/Test/Simple.pm line 77. # Looks like you failed 1 test of 1. -------------------------- Cut here ---------------------------- Is this just the way it is? I would have thought that 50% self-test failure would be ringing alarm bells everywhere if it were common, so I can only conclude that there's something odd about my environment, or that I'm doing something wrong, but this is about as vanilla a build process as I can possibly make it. I follow the steps for Win32 in INSTALL, and as I said at the start of this message, the nmake process goes cleanly, not a single warning or error. The ONLY non-standard thing I do is change the /MD switch (link to the DLL versions of the runtime libraries) to /MT (static link the runtimes) because I don't want to have external dependencies in my production environments (I lived in "DLL Hell" for so long that I'm now quite paranoid about that). This change has never caused problems in the past, and doesn't seem to be relevant to the problems I'm seeing. I've been building OpenSSL myself for a number of years, most recently with the end-of-life v1.0.2 builds, which always go without a hitch. As I remarked, I've been putting off moving to v1.1.1 because I'm so uneasy about these self-test failures, but I can't continue doing that any longer as TLS3 starts coming on stream. Anyone have any insights into what I'm doing wrong, or what I can do about this? I'm very reluctant to use the software in production if it can't pass its own self-test regime, even if it appears to work normally otherwise. Comments most welcome. Cheers! -- David --
