On Thu, Nov 25, 2010, Robert Dugal wrote: > I am developing an ENGINE for OpenSSL 1.0.0a and 0.9.8o. Among other things > this engine implements digest methods like SHA1. > > While testing this engine I discovered memory leaks when using > PEM_write_bio_PKCS8PrivateKey(). > I traced the leak back to the inner and outer SHA1 digest contexts used for > HMAC in PKCS5_PBKDF2_HMAC(). > The leak is related to the md_data in the EVP_MD_CTX. My engine saves it's > own data in ctx->md_data which includes allocation of additional pointers. > That data gets initialized by my ENGINE implementation of the init() callback > and destroyed by the cleanup() callback. > One thing I have noticed about the ENGINE callback design is that the init() > callback cannot check the md_data to see if it was previously already > initialized. > OpenSSL doesn't initialize the memory to 0 when md_data is allocated. So my > init() callback has no choice but to assume that the md_data is uninitialized. > So if the init() callback is called more than once without calling the > cleanup() callback my ENGINE implementation will introduce a memory leak. > This is where the source of the problem lies. > > In the code below the "while(tkeylen)" loop may result in multiple passes of > the HMAC code. On each pass it does HMAC_Init_ex(),HMAC_Update(),HMAC_Final(). > HMAC_CTX_cleanup() is only called once after exiting the while loop. > > Each time HMAC_Init_ex() is called my ENGINE SHA1 init() callback is called > and it initializes the md_data for the inner and outer digest contexts. > But HMAC_Final() never calls EVP_DigestFinal_ex() or EVP_MD_CTX_cleanup() on > these inner and outer digest contexts. > Because these inner/outer digest context are not finalized or cleaned up, > each pass through the loop results in them being re-initialized by my init() > callback and memory leaks introduced. > > If I modify HMAC_Final() to call EVP_MD_CTX_cleanup(&ctx->i_ctx) and > EVP_MD_CTX_cleanup(&ctx->o_ctx), then my memory leaks go away. > So either there is a bug in OpenSSL or I am misunderstanding the ENGINE > digest interfaces. >
The auto allocation of md_data is a convenience measure which reduces code duplication in some cases (for example software based implementations). The number of cleanup calls is kept to a minimum to reduce memory fragmentation by allowing contexts to be reused. In some cases (such as yours) the default behaviour isn't appropriate. The amount of memory auto allocated is determined by the ctx_size field of the EVP_MD structure. Have you tried setting this to zero? That should shut off the auto allocation and freeing and leave the implementation to handle it in its init() and cleanup() callbacks. Steve. -- Dr Stephen N. Henson. OpenSSL project core developer. Commercial tech support now available see: http://www.openssl.org ______________________________________________________________________ OpenSSL Project http://www.openssl.org Development Mailing List [email protected] Automated List Manager [email protected]
