If you are facing memory fragmentation issues, try to use a memory pool: Allocate a huge bunch of memory in small blocks that are managed by a (for example) circular list, and then just use blocks from that list (for example see http://kitt3n.homeftp.net/?p=11).
/Thomas 2010/2/18 ghazi bousselmi <[email protected]>: > Hi, > Memory fragmentation is an important problem for applications that use > memory intensively (alloc and free) with small and numerous chunks. > We have faced this problem for our product, where the application has huge > free memroy (above 700 MB), but cannot allocate any piece of certain > size. We have solved it by developing a memory manager that reduces memory > management to virtually 0. > I am not allowed to say much about it, but your idea is right: supplant the > memory allocator of the runtime library, and allocate yourself huge memory > blocs, from which you will give small pieces to whoever is needing it. > Regrads > > 2010/2/18 mobi phil <[email protected]> >> >> I hope you do not feel polluted by my emails :) >> >> In all C++ projects (and not only C++) I faced sooner or later memory >> fragmentation. Given the specificity of WT application, I think one >> could do some minimal trick to reduce significantly memory >> fragmentation, improving thus performance of the application. The >> linux libc does not have the best possible allocator, especially for >> small objects, and applications can have their memory very fast >> defragmented, given that C++ allocates often lots of small objects. >> Especially in multi threaded applications this can be a serious >> performance penalty. >> >> What is specific for Wt is that when a session is terminated, all the >> object tree is deleted. Now if one would allocate with malloc() an >> area of memory and would manage itself that memory and would free() >> that area after the session of memory, the total defragmentation would >> be zero. What is important, is that you need only basic memory >> management above the area allocated with malloc. Implementation of the >> new operator for size X would return a pointer to current p, and >> current will become current+X. On delete you would not do anything. >> This game could be generalized in case that the same thread supports >> several sessions, or several sessions are served by several threads: >> >> 1. there would be a global map session id to area, that would help new >> to go to the right memory area. Each thread would have such a map, so >> it would not be necessary to do extra locking. >> >> 2. If a session uses two threads then locking during allocation could >> be also avoided, as each thread would have a map as above, the only >> question is how to go to the right area based on the thread ID. >> >> One could limit thus the memory usage of a session. If a user would >> use to many sessions, you either allocate new blocks for it, or raise >> error "dear user, you are too expensive for me :)" >> >> Obviously this would not fit for long sessions, or where the memory >> consumption of a session is considerably high. >> >> >> rgrds, >> mobi phil >> >> being mobile, but including technology >> http://mobiphil.com >> >> >> ------------------------------------------------------------------------------ >> Download Intel® Parallel Studio Eval >> Try the new software tools for yourself. Speed compiling, find bugs >> proactively, and fine-tune applications for parallel performance. >> See why Intel Parallel Studio got high marks during beta. >> http://p.sf.net/sfu/intel-sw-dev >> _______________________________________________ >> witty-interest mailing list >> [email protected] >> https://lists.sourceforge.net/lists/listinfo/witty-interest > > > > -- > Ghazi Bousselmi > Accent Circonflexe Groupe BV > Nieuwezijds Voorburgwal 4-10 > 1012 RZ Amsterdam > > Office: +31 (0)20 524 0400 > Fax: +31 (0)20 427 1402 > > Email: [email protected] ; [email protected] > Website: www.accentgroupe.com ; http://ghazi.bousselmi.googlepages.com > > > ------------------------------------------------------------------------------ > Download Intel® Parallel Studio Eval > Try the new software tools for yourself. Speed compiling, find bugs > proactively, and fine-tune applications for parallel performance. > See why Intel Parallel Studio got high marks during beta. > http://p.sf.net/sfu/intel-sw-dev > _______________________________________________ > witty-interest mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/witty-interest > > ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev _______________________________________________ witty-interest mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/witty-interest
