===========================================================
Summary - openssl-0.9.8 "make test" failure for SunOS 4.1.4
===========================================================
OpenSSL version: 0.9.8a-dev [openssl-0.9.8-stable-SNAP-20050906]
Options: no-gmp no-krb5 no-mdc2 no-rc5 no-shared no-threads no-zlib
no-zlib-dynamic
OS (uname): SunOS mfg 4.1.4 1 sun4m
OS (config): sun4m-sun-sunos4
Target (default): sunos-gcc
Target: sunos-gcc
Compiler: gcc version 2.8.1 (also 2.7.2.3) (also optimization off)
Failing step from "make test_gen" test/gentest script:
mfg(259) ../apps/openssl req -config test.cnf -verify -in testreq.pem -noout
verify failure
6901:error:04077064:rsa routines:RSA_verify:algorithm mismatch:rsa_sign.c:211:
6901:error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP
lib:a_verify.c:168:
The culprit turns out to be "memcmp". In SunOS 4.x, "memcmp" implements
a signed byte comparision implied by the native data type "char".
Naturally this is at odds with current standards calling for an unsigned
byte comparison memcmp. The BUGS section for the SunOS 4.1.4 "memcmp"
man page kindly warns about the signed versus unsigned implementation
dependent conflict.
Most of OpenSSL's 127 calls to "memcmp" occurring in 44 different files
are simply tests for equality (and 13 of the files checking only equality
are test programs). These equality checks work fine using the SunOS
signed "memcmp". However, there are 9 memcmp calls in 7 files which
occur in comparison functions which could be used for ordering purposes
(specifically, memcmp's return value is the return value of the function).
My first fix version inserted a macro replacement in these 7 files
and left the other equality check memcmp calls unchanged. But it is simpler
and works just fine to add the memcmp macro replacement to the pile of
other SunOS 4.x macro replacements in "e_os.h". However, one of the 7
target files is missing an explicit or implicit inclusion of e_os.h and
thus gains an inclusion of "e_os.h" via the patch. (I first added
"cryptlib.h" to be consistent with the other implicit inclusions of
e_os.h in crypto/asn1 source files, but then reverted to minimalism.)
1) crypto/asn1/a_set.c
2) crypto/asn1/asn1_lib.c
3) crypto/asn1/tasn_enc.c <== this one was missing an e_os.h inclusion
4) crypto/objects/obj_dat.c
5) crypto/objects/obj_lib.c
6) crypto/x509/x509_cmp.c
7) ssl/ssl_lib.c
Of the rest of the memcmp containing files, a visual survey found 7 which
are missing e_os.h explicitly or implicitly, but since they just use
memcmp for equality checks it don't matter anyway. Turns out having the
macro definition in e_os.h only adds a few hundred bytes to libcrypto.a which
was nearly 2 megabytes anyway. (and a similar small amount to libssl.a).
Note that a "signed memcmp" by itself is not necessarily evil. For
example, the "problem" comparison function "obj_cmp()" of
crypto/objects/obj_dat.c with a "signed memcmp" still defines a total
ordering on byte data, albeit a different one than when using an
"unsigned memcmp". Either should work if used exclusively for
building and searching lookup tables, but likely with differing
execution paths.
One example problem for openSSL occurs because the lookup table defined by
the array "static ASN1_OBJECT *obj_objs[NUM_OBJ]" of crypto/objects/obj_dat.h
is precomputed by the perl script obj_dat.pl which uses the perl comparison
sub "obj_cmp" which constructs the same ordering as would the C code
comparison function obj_cmp() of crypto/objects/obj_dat.c assuming the use of
an "unsigned memcmp". (The underlying perl "cmp" function is not affected
by the "signed memcmp" under SunOS 4.x., so rebuilding obj_dat.h with an
unmodified obj_dat.pl with a SunOS perl doesn't "fix it" or change it in
any significant way) - besides that approach would be too risky anyway
as there might be other dependencies. I'll admit I was and may yet fool
around with that as a curiosity exercise :)
The DETAILS section demonstrates the failure which occurs when the following
two hex-encoded DER character strings from crypto/objects/obj_dat.h are
compared during a table lookup on the obj_objs array.
70 static unsigned char lvalues[5002]={
...
123 0x2B,0x0E,0x03,0x02,0x1A, /* [361] OBJ_sha1 */
...
740 0x2B,0x81,0x04,0x00,0x08, /* [4806] OBJ_secp160r1 */
Although the existence of this "memcmp" problem certainly existed in past
versions of openSSL under SunOS 4.x, it did not break "make test" until
the 0.9.8 addition of elliptic curve entries in objects.txt et al induced
the commonly occurring "sha1" lookup to fail (e.g. above and DETAILS).
The required TSU notification containing the attached patch has been sent
to [EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED]
russell ruby - russ at sludge.net
===========================================================
DETAILS
===========================================================
"make test" fails - for example, the script "test/testgen" shows errors on
the verify check:
# cd test
# ../apps/openssl req -config ./test.cnf -verify -in ./testreq.pem -noout
verify failure
17451:error:04077064:rsa routines:RSA_verify:algorithm mismatch:rsa_sign.c:211:
17451:error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP
lib:a_verify.c:168:
The SunOS produced "testreq.pem" carried over to a Linux box verifies Okay,
and a testgen "testreq.pem" file created on the Linux box fails on SunOS.
# So given these clues, run gdb in parallel on "openssl-0.9.8/apps/openssl"
# on both Linux and SunOS using the same testreq.pem from SunOS.
(gdb) break main
(gdb) break rsa_sign.c:RSA_verify
(gdb) req -config ../test.cnf -verify -in ../testreq.pem
# The behaviour agrees during ./crypto/rsa/rsa_sign.c until after entering:
=> sigtype=OBJ_obj2nid(sig->algor->algorithm); (line 188 rsa_sign.c)
-------------------------------------------------
# this initiates a binary search with search key "+\016\003\002\032" (which
# is the ./crypto/objects/obj_dat.h "lvalues" array entry for "OBJ_sha1")
# obj_objs from obj_dat.h provides the lookup table.
# obj_cmp from objects.c is the comparison function.
=> op=(ASN1_OBJECT **)OBJ_bsearch((const char *)&a,(const char *)obj_objs,
NUM_OBJ, sizeof(ASN1_OBJECT *),obj_cmp);
(this statement from: line 385 in ./crypto/objects/obj_dat.c)
-------------------------------------------------
# (OBJ_bsearch is a wrapper for OBJ_bsearch_ex)
# After several iterations of the binary search while loop have completed
# in agreement on the two machines, the behaviour diverges - toward success
# on the Linux box and failure on the SunOS box.
#
# In ./crypto/objects/obj_dat.c:OBJ_bsearch_ex (line 566)
const char *OBJ_bsearch_ex(const char *key, const char *base, int num,
int size, int (*cmp)(const void *, const void *), int flags)
{
int l,h,i=0,c=0;
const char *p = NULL;
if (num == 0) return(NULL);
l=0;
h=num;
while (l < h)
{
i=(l+h)/2;
p= &(base[i*size]);
=> c=(*cmp)(key,p); (line 579 in obj_dat.c)
if (c < 0)
h=i;
else if (c > 0)
l=i+1;
else
break;
}
--------------------------------------------------
# The diverging behaviour shows up here, during "(*cmp)"
# (which refers to obj_cmp, also in crypto/objects/obj_dat.c line 549)
static int obj_cmp(const void *ap, const void *bp)
{
int j;
const ASN1_OBJECT *a= *(ASN1_OBJECT * const *)ap;
const ASN1_OBJECT *b= *(ASN1_OBJECT * const *)bp;
j=(a->length - b->length);
if (j) return(j);
=> return(memcmp(a->data,b->data,a->length)); (line 1265 obj_dat.c)
}
(gdb) p *a (search key variable)
$4 = {sn = 0x0, ln = 0x0, nid = 0, length = 5,
data = 0x290908 "+\016\003\002\032", flags = 9}
(gdb) p *b (current comparison target from lookup table)
$5 = {sn = 0xb4088 "secp160r1", ln = 0xb4088 "secp160r1", nid = 709,
length = 5, data = 0x239bae "+\201\004", flags = 0}
# These data values are defined by the lvalues hex encoded entries of obj_dat.h
# "+\016\003\002\032" <==> 0x2B,0x0E,0x03,0x02,0x1A, /*[361] OBJ_sha1*/
# "+\201\004\000\010" <==> 0x2B,0x81,0x04,0x00,0x08, /*[4806] OBJ_secp160r1*/
# Note the "negative" second byte for SunOS signed char, so checking memcmp:
# On SunOS memcmp decides: a->data > b->data using the signed memcmp
(gdb) p memcmp(a->data,b->data,a->length) (on SunOS)
$6 = 141
# While Linux memcmp decides: a->data < b->data using an unsigned memcmp
(gdb) p memcmp(a->data,b->data,a->length) (on slackware Linux)
$4 = -1
# SunOS takes the wrong branch in in OBJ_bsearch_ex and fails to find
# a match, resulting in the error:
verify failure
9737:error:04077064:rsa routines:RSA_verify:algorithm mismatch:rsa_sign.c:217:
9737:error:0D0C5006:asn1 encoding routines:ASN1_item_verify:EVP
lib:a_verify.c:168:
=========================================================================
Here is a patch which adds an unsigned byte macro replacement for memcmp
to e_os.h, along with an inclusion of "e_os.h" to crypto/ansn1/tasn_enc.c.
------------------------ Cut here ---------------------------------
diff -Naur openssl-0.9.8-stable-SNAP-20050906-orig/crypto/asn1/tasn_enc.c
openssl-0.9.8-stable-SNAP-20050906-work/crypto/asn1/tasn_enc.c
--- openssl-0.9.8-stable-SNAP-20050906-orig/crypto/asn1/tasn_enc.c
2004-04-25 05:46:39.000000000 -0700
+++ openssl-0.9.8-stable-SNAP-20050906-work/crypto/asn1/tasn_enc.c
2005-09-06 10:28:21.000000000 -0700
@@ -59,6 +59,7 @@
#include <stddef.h>
#include <string.h>
+#include "e_os.h"
#include <openssl/asn1.h>
#include <openssl/asn1t.h>
#include <openssl/objects.h>
diff -Naur openssl-0.9.8-stable-SNAP-20050906-orig/e_os.h
openssl-0.9.8-stable-SNAP-20050906-work/e_os.h
--- openssl-0.9.8-stable-SNAP-20050906-orig/e_os.h 2005-08-02
16:03:17.000000000 -0700
+++ openssl-0.9.8-stable-SNAP-20050906-work/e_os.h 2005-09-06
10:28:21.000000000 -0700
@@ -565,6 +565,13 @@
extern char *sys_errlist[]; extern int sys_nerr;
# define strerror(errnum) \
(((errnum)<0 || (errnum)>=sys_nerr) ? NULL : sys_errlist[errnum])
+ /* SunOS 4.x signed memcmp breaks table lookup of ASN1_OBJECT *obj_objs.
+ This replacement assumes gcc statement as expression extension. */
+#define memcmp(s1,s2,n) ({int _ii,_uclim,_ucdiff=0; \
+ unsigned char *_ucp1=(unsigned char *)(s1), *_ucp2=(unsigned char *)(s2); \
+ for (_ii=0,_uclim=(n); _ii<_uclim; _ii++) \
+ if (_ucdiff=_ucp1[_ii] - _ucp2[_ii]) break; \
+ _ucdiff; })
#endif
#ifndef OPENSSL_EXIT
______________________________________________________________________
OpenSSL Project http://www.openssl.org
Development Mailing List [email protected]
Automated List Manager [EMAIL PROTECTED]
