> From: owner-openssl-us...@openssl.org On Behalf Of Ryan Smith
> Sent: Friday, 02 July, 2010 18:31
> I have written a FIPS-1.1.2 compliant (OpenSSL 0.9.7m) application
> that validates certificates that are read in from files. It also loads
> the CA certificates and corresponding CRLs from files. I am trying to
> determine any limitations with loading large CRLs (~200-250 MB) and to
Not your actual problem but IMNVHO any CA that needs to issue CRLs
anywhere near that large should not be trusted in the first place.
> characterize the resulting performance. I did not have any problem
> using CRLs that are ~~100MB in size or smaller. However with the ~200MB
CRL,
> I get the following error [in PEM_read_bio_X509_CRL]:
> 1418976:error:0D078064:asn1 encoding routines:ASN1_ITEM_EX_D2I:aux
error:tasn_dec.c:407:Type=X509_CRL_INFO
> 1418976:error:0D08303A:asn1 encoding
routines:ANS1_TEMPLATE_D2I:nested asn1 error:tasn_dec.c:567:Field=crl,
Type=X509_CRL
> 1418976:error:0906700D:PEM routines:PEM_ASN1_read_bio:ASN1
lib:pem_oth.c:82:
The only likely cause I see is the call to asn1_enc_save (line 397 in
0.9.7m).
That needs to allocate enough memory to copy the entire DER, and would fail
if the heap aka arena does not have sufficient available (contiguous) space.
> The certificates and ~200MB CRL in question are all validated
successfully
> using the OpenSSL command line
> openssl verify -CAfile <cafile> -crl_check <certificates>
1. I wouldn't trust that definitely means no error, without checking. I have
seen
some commandline logic suppress errors it thinks are 'irrelevant' or
'unimportant'.
2. That's definitely a different executable, and may have been linked
differently
or have a different memory usage pattern. Does your program
CRYPTO_set_mem_functions
or CRYPTO_malloc_debug_init ? Do you have lots of other dynamic space
allocated,
or had space previously allocated and free'd in a way that leaves
fragmentation?
Does your program run in an environment that has a (stricter) quota on
memory?
2a. It looks to me like PEM_read_bio uses BUF_MEM_grow in a way that may
fragment.
Can you try, just before PEM_read__CRL, malloc'ing and free'ing at least two
and
I think three blocks of the size of the DER that will be implictly read plus
some
slop, say (filesize-60)/4*3+100 assuming the file contains only the PEMmed
CRL?
Hmmm. These particular items (CRLs) will not be reencoded, so the enc_save
looks unnecessary. It might be nice if this were somehow optional; as above,
I don't think CRLs should be so big this is an issue, but some other things
using (I assume) the same ASN1 mechanism like PKCS7 and mime reasonably
might.
______________________________________________________________________
OpenSSL Project http://www.openssl.org
User Support Mailing List openssl-users@openssl.org
Automated List Manager majord...@openssl.org
