Bob, Just a little more explanation here. Suppose we have a bunch of structures allocted into the pages of memory and a connection goes sour and is timing out. Suppose our server is under substantial memory pressure. In a situation like this if _any_ structure on any active connection happens to live in a memory page otherwise or even largely used by the dead connection, then the machine CANNOT swap that page. The way malloc() works this will be the normal situation because stuctures are alloacted temporaly (in the time domain) and by size. It becomes a real problem when many small structs are malloc()ed.
next, in order to optimize allocation and freeing malloc() actually does quite a lot of work. When I wrote that poor boy code I looked for as simple a solution as possible. In OpenSSL I would make the assumption that the high water mark on pretty much any connection will be about as high as the high water mark on any other. This is measured by the total amount of memory needed for all structures relavent to the given connection. But if this is not so its not really all that terrible because if the pool of pages is reused then it just grows back into the old page set as required. Memory pages sitting in a swap file are something I consider free. It might require more I/O to free them up than to just let them fester on disk. Note that I didn't bother with a free space chain. Depending on the nature of the code if something like the pfa_blah() functions are to be used then a free chain might be necessary and it might make sense to organise some sort of a stack as well. Maybe they did this. I never looked. What I am thinking here is that whenever a function is called it gets a stack marker and automatic variables are allocated into the stack... but in a forked or threaded model I'm not sure if each fork() or thread() gets its own stack and how this might interact with a pfa_blah() scheme indexed by an fd. Regardless if an fd is not known and is not relevent to a function then I suppose one could index via a pid? This of course depends on the specfic way the code is written and a lot of assumptions may not be valid. To me however I think one observation has to be correct but this will have to be confirmed. We should be able to collect all malloc()s into a pool of pages organised around an fd which is what the operating system uses to identify the connection. If so then we should NOT need to care if that fd is alive or dead. Somehow the o/s and the code knows about this and hopefully will eventually close the file if the connection dies. regardless of how this might happen. If a connection dies then the O/S will collect all control blocks associated with the fd and somehow make the memory and the fd number available for another connections. What this means is that in the code, select() will re-use a dead fd#. If so then all one really needs to do is call pfa_init() at the time one calls fopen(). pfa_init() will reset the page pool for the fd or initializes one if there is none there. Note the maximum number of file descriptors is FD_SETSIZE which is defined in <sys/select.h>. So it should work and should be easy to layer in. On Fri, Jun 24, 2011 at 04:35:57PM -0700, Yan, Bob wrote: > Thank you and Eric for the inputs. I will look at them and see what I should > do with this. > > Have a nice weekend. > Bob > > > -----Original Message----- > From: owner-openssl-us...@openssl.org > [mailto:owner-openssl-us...@openssl.org] On Behalf Of t...@terralogic.net > Sent: Friday, June 24, 2011 4:52 PM > To: openssl-users@openssl.org > Subject: Re: Question regarding to memory leak > > I have suggested this before. Write your own memtools. > > > http://www.terralogic.net/developer/developer.html > > > I tossed up a couple poor boy examples. > > Note the calls: > > struct pfa_ControlBlock chain1 = { ipfa_ControlBlock } > , chain2 = { ipfa_ControlBlock }; > > pfa_Init( &chain1, 8172 ); > pfa_Init( &chain2, 8172 ); > > p = pfa_alloc( &chain1, 10, &ierr ); > > > > > In OpenSSL for instance each connection is identified by an fd (for the > socket) and there are a fixed number of these. > > One can set up an array of pointers and allocate a page of memory for each > connection at the time the connection is opened. > > One wants a 1:1 correspondence between the fd flags from select() and the > pfa_Control_Block pointer array. > > Then when _ANY_ memory is needed simply use pfa_alloc() with the approriate > control block of which the address can easily be looked up from the pointer > array. > > At any time if the connection is lost... forget it. Close the socket and the > next time the fd is used the old pages will be fetched from virtual memory as > required. No leaks! Also all memory for a connection will be in a small > group of pages which will be flushed to the swap file and they will stay > there because there will never be any competing connections using that pool > of pages! (malloc will shuffle memory for many connections into a given set > of pages). > > > If we want to actally free the memory because we are closing the fd then its > simple... follow the linked list and free the pages. > > > ------------- > > Next it really doesn't matter if malloc and pfa_alloc() are both used. Any > structres which use pfa_blah will live in a world of their own which is > totally compatible with anything malloc() might do. We just wouldn't want a > struct allocated with pfa_blah() pointed to by something allocated with > malloc() because if we call pfa_init() then all those pointers in the > malloc()ed structs are invalid. However by definition if we call pfa_init() > then that fd is dead so anything that points to anything allocated for that > dead fd is likely invalid anyways. > > > The thing is this. One can free and reallocate in a pfa_blah page pool. I > didn't need to do this so I never bothered. But it can be done. Anything > which is not actually freed can be re-initialized at any time by calling > pfa_init(). At that point all pages are wiped and ready for re-use. One > doesn't have to go through all sorts of linked structures and determine peice > by piece what was and what was not allocated and whether something has been > freed. There is one master and simple linked list structure and that is > found from the fd which identifies the socket. > > > ------------- > > As I said I offered this a few years ago. I was told there is already some > sort of memory management being used. But I keep seeing comments on leaks so > I'm not sure what is done because I think if one uses the fd one can sort of > allocate a pool of pages for each connection and as long as all malloc() is > done within that pool then we shouldn't have leaks. > > Also if all malloc()s for an fd are in their own pool of pages then the > machine can swap that connection out if required and if the connection times > out and dies jsut re-initialize it when the fd is re-opened (on another > connection). > > > On Fri, Jun 24, 2011 at 01:20:42PM -0700, Eric S. Eberhard wrote: > > As a general comment not all memory leaks reported by these tools are > > a bad thing. I often write code that has these type of leaks on > > purpose for performance reasons. For example a function that is > > called often and malloc's memory ... rather than malloc and free each > > time (causing context switching and generally slow) I just make the > > pointer static and a size variable static. I use the pointer until it > > is too small, then I realloc to a larger size. In modern systems > > often the "leak" is worth the performance gain. I run on IBM AIX. > > Having said that, I have not dug in to your specifics which may just > > be bugs, an error with the tool, or deliberate. Eric > > > > > > At 11:58 AM 6/24/2011, Yan, Bob wrote: > > >Hi, > > > > > >I have used IBM purify to check my test program which invokes openssl > > >library. There are some memory leaks reported by Purify, please see > > >below. Could somebody point to me from which function those leaks > > >were generated, and how to avoid those leaks? Thanks, Bob > > > > > > > > > MLK: 1104 bytes leaked in 46 blocks > > > This memory was allocated from: > > > malloc [rtlib.o] > > > CRYPTO_malloc [libcrypto.so.1.0.0] > > > ASN1_STRING_type_new [libcrypto.so.1.0.0] > > > ASN1_primitive_new [libcrypto.so.1.0.0] > > > asn1_item_ex_combine_new [libcrypto.so.1.0.0] > > > asn1_item_ex_combine_new [libcrypto.so.1.0.0] > > > ASN1_item_ex_d2i [libcrypto.so.1.0.0] > > > asn1_template_noexp_d2i [libcrypto.so.1.0.0] > > > Block of 24 bytes (46 times); last block at 0x2aaaac0f7218 > > > MLK: 836 bytes leaked in 2 blocks > > > This memory was allocated from: > > > malloc [rtlib.o] > > > CRYPTO_malloc [libcrypto.so.1.0.0] > > > asn1_enc_save [libcrypto.so.1.0.0] > > > ASN1_item_ex_d2i [libcrypto.so.1.0.0] > > > asn1_template_noexp_d2i [libcrypto.so.1.0.0] > > > asn1_template_ex_d2i [libcrypto.so.1.0.0] > > > ASN1_item_ex_d2i [libcrypto.so.1.0.0] > > > ASN1_item_d2i [libcrypto.so.1.0.0] > > > Block of 432 bytes at 0x2aaaac093668 > > > Block of 404 bytes at 0x2aaaac009d58 > > > MLK: 800 bytes leaked in 20 blocks > > > This memory was allocated from: > > > malloc [rtlib.o] > > > CRYPTO_malloc [libcrypto.so.1.0.0] > > > ASN1_OBJECT_new [libcrypto.so.1.0.0] > > > c2i_ASN1_OBJECT [libcrypto.so.1.0.0] > > > asn1_ex_c2i [libcrypto.so.1.0.0] > > > asn1_d2i_ex_primitive [libcrypto.so.1.0.0] > > > ASN1_item_ex_d2i [libcrypto.so.1.0.0] > > > asn1_template_noexp_d2i [libcrypto.so.1.0.0] > > > Block of 40 bytes (20 times); last block at 0x2aaaac097a78 > > > MLK: 600 bytes leaked in 15 blocks > > > This memory was allocated from: > > > malloc [rtlib.o] > > > CRYPTO_malloc [libcrypto.so.1.0.0] > > > asn1_item_ex_combine_new [libcrypto.so.1.0.0] > > > ASN1_item_ex_d2i [libcrypto.so.1.0.0] > > > asn1_template_noexp_d2i [libcrypto.so.1.0.0] > > > asn1_template_ex_d2i [libcrypto.so.1.0.0] > > > ASN1_item_ex_d2i [libcrypto.so.1.0.0] > > > asn1_template_noexp_d2i [libcrypto.so.1.0.0] > > > Block of 40 bytes (15 times); last block at 0x2aaaac0f6e98 > > > MLK: 376 bytes leaked in 33 blocks > > > This memory was allocated from: > > > malloc [rtlib.o] > > > CRYPTO_malloc [libcrypto.so.1.0.0] > > > ASN1_STRING_set [libcrypto.so.1.0.0] > > > asn1_ex_c2i [libcrypto.so.1.0.0] > > > asn1_d2i_ex_primitive [libcrypto.so.1.0.0] > > > ASN1_item_ex_d2i [libcrypto.so.1.0.0] > > > asn1_template_noexp_d2i [libcrypto.so.1.0.0] > > > asn1_template_ex_d2i [libcrypto.so.1.0.0] > > > Block of 14 bytes (21 times); last block at 0x2aaaac0f7168 > > > Block of 12 bytes (2 times); last block at 0x2aaaac096828 > > > Block of 11 bytes (2 times); last block at 0x2aaaac096338 > > > Block of 5 bytes (6 times); last block at 0x2aaaac0976f8 > > > Block of 3 bytes (2 times); last block at 0x2aaaac095e58 > > > . > > > . > > > . > > > > > > > > > > > > Eric S. Eberhard > > (928) 567-3727 Voice > > (928) 567-6122 Fax > > (928) 301-7537 Cell > > > > Vertical Integrated Computer Systems, LLC Metropolis Support, LLC > > > > For Metropolis support and VICS MBA Support!!!! http://www.vicsmba.com > > > > Pictures of Snake in Spring > > > > http://www.facebook.com/album.php?aid=115547&id=1409661701&l=1c375e1f4 > > 9 > > > > Pictures of Camp Verde > > > > http://www.facebook.com/album.php?aid=12771&id=1409661701&l=fc0e0a2bcf > > > > Pictures of Land Cruiser in Sedona > > > > http://www.facebook.com/album.php?aid=50953&id=1409661701 > > > > Pictures of Flagstaff area near our cabin > > > > http://www.facebook.com/album.php?aid=12750&id=1409661701 > > > > Pictures of Cheryl in a Horse Show > > > > http://www.facebook.com/album.php?aid=32484&id=1409661701 > > > > > > Pictures of the AZ Desert > > > > http://www.facebook.com/album.php?aid=58827&id=1409661701 > > > > (You can see why we love this state :-) ) > > > > > > > > > > > > > > > > > > ______________________________________________________________________ > > OpenSSL Project http://www.openssl.org > > User Support Mailing List openssl-users@openssl.org > > Automated List Manager majord...@openssl.org > ______________________________________________________________________ > OpenSSL Project http://www.openssl.org > User Support Mailing List openssl-users@openssl.org > Automated List Manager majord...@openssl.org > ______________________________________________________________________ > OpenSSL Project http://www.openssl.org > User Support Mailing List openssl-users@openssl.org > Automated List Manager majord...@openssl.org ______________________________________________________________________ OpenSSL Project http://www.openssl.org User Support Mailing List openssl-users@openssl.org Automated List Manager majord...@openssl.org
