Hi,
I tried tcmalloc but the results are the same. RSS memory usage is
even slightly higher when I used tcmalloc
RSS after main initialization
glibcMalloc : 2080KB
tcMalloc: 2756KB
RSS after XML node initialization
glibcMalloc : 7004KB
tcMalloc: 7560KB
RSS after removing the XML node
glibcMalloc : 7068KB
tcMalloc: 7764KB
I also created a small test program in which I manage to reproduce the
same results:
#include <stdlib.h>
#include <stdio.h>
#include <string>
using namespace std;
int main()
{
system("top -b -n 1 > result_top.txt");
int **t = new int*[100000];
for(long i = 0; i < 100000; i++)
{
t[i] = new int();
}
system("top -b -n 1 >> result_top.txt");
for(long i = 100000 - 1; i >= 0; i--)
{
delete t[i];
}
delete [] t;
system("top -b -n 1 >> result_top.txt");
return 0;
}
Top output after main initialization:
glibc: 2716
tcMalloc: 4948
Top output after “t” array initialization:
glibc: 2572
tcMalloc: 2668
Top output after deleting the “t” array:
glibc: 2352
tcMalloc: 2708
Br,
Jakub
On 14 February 2012 20:20, Howard Chu <[email protected]> wrote:
> John Alvord wrote:
>>
>> I am an interested bystander. Last year I diagnosed a 2+ year long storage
>> growth issue where there was no storage leak... firmly proven.
>>
>> The diagnosis was a case of extreme fragmentation. The process needed
>> large
>> chunks of contiguous storage [2-3 megs] and when released the system
>> storage
>> allocator [AIX] did not segregate the large chunks. Smaller requests
>> populated
>> the newly freed area "sometimes" and thus a new storage chunk was needed.
>> When
>> there was no storage available for a large chunk, the process space was
>> increased in a large chunk. Repeat until the upper limit of a 32 bit AIX
>> process size [roughly 2 gigs] is exceeded and the process failed, which
>> took
>> 10-12 days.
>>
>> Not sure how you could prove that case - in effect would need to look at
>> the
>> physical layout of allocated and free storage.
>
>
> We see this all the time in OpenLDAP slapd with the glibc allocator. I've
> documented it pretty extensively. Google tcmalloc is much less susceptible
> to the problem.
>
> http://highlandsun.com/hyc/malloc/
>
>>
>> Just one possibility...
>>
>> John Alvord, IBM
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: *Jakub Kubinski* <[email protected]
>> <mailto:[email protected]>>
>> Date: Tue, Feb 14, 2012 at 3:23 AM
>> Subject: [Valgrind-users] RSS memory is increasing but memory leaks are
>> not
>> reported
>> To: [email protected]
>> <mailto:[email protected]>
>>
>>
>> Hello,
>>
>> I have a "strange" memory leak in my XML parser which I cannot locate
>> using valgrind.
>>
>> In my testcase I'm creating and deleting a XML node and after the test
>> valgrind is not showing any memory leaks but rss memory is still
>> active VmRSS: 11184 kB ( VmRSS after the XML node creation was
>> 11048 kB so it increased by 136 after executing the XML node
>> destructor )
>>
>> In the valigirng report I found "--30004-- memcheck: auxmaps: 179
>> auxmap entries (11456k, 11M) in use" which fits to the remaining 11048
>> kB in the RSS memory.What does this mean ?
>>
>>
>>
>> ==30004== Memcheck, a memory error detector.
>> ==30004== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
>> ==30004== Using LibVEX rev 1658, a library for dynamic binary translation.
>> ==30004== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
>> ==30004== Using valgrind-3.2.1, a dynamic binary instrumentation
>> framework.
>> ==30004== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
>> ==30004==
>> ==30004== My PID = 30004, parent PID = 27692. Prog and args are:
>> ==30004== ./ala
>> ==30004==
>> --30004--
>> --30004-- Command line
>> --30004-- ./ala
>> --30004-- Startup, with flags:
>> --30004-- --tool=memcheck
>> --30004-- --leak-check=full
>> --30004-- --leak-resolution=high
>> --30004-- --num-callers=40
>> --30004-- --show-reachable=yes
>> --30004-- --error-limit=no
>> --30004-- -v
>> --30004-- --log-file=valgrind.log
>> --30004-- Contents of /proc/version:
>> --30004-- Linux version 2.6.39.4-1.NSN.kiuas
>> ([email protected] <mailto:[email protected]>)
>> (gcc
>>
>> version 4.1.2 20080704 (Red Hat
>> 4.1.2-46)) #1 SMP Wed Aug 10 12:16:36 EEST 2011
>> --30004-- Arch and hwcaps: AMD64, amd64-sse2
>> --30004-- Valgrind library directory: /usr/lib64/valgrind
>> --30004-- Reading syms from /home/kubinski/max/22/ala (0x400000)
>> --30004-- Reading syms from /usr/lib64/valgrind/amd64-linux/memcheck
>> (0x38000000)
>> --30004-- object doesn't have a dynamic symbol table
>> --30004-- Reading syms from /lib64/ld-2.5.so <http://ld-2.5.so>
>> (0x3734000000)
>>
>> --30004-- Reading suppressions file: /usr/lib64/valgrind/default.supp
>> --30004-- Reading syms from
>> /usr/lib64/valgrind/amd64-linux/vgpreload_core.so (0x4802000)
>> --30004-- Reading syms from
>> /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so (0x4A03000)
>> --30004-- REDIR: 0x3734014400 (index) redirected to 0x4A06550 (index)
>> --30004-- REDIR: 0x37340145B0 (strcmp) redirected to 0x4A067D0 (strcmp)
>> --30004-- REDIR: 0x37340145E0 (strlen) redirected to 0x4A06700 (strlen)
>> --30004-- Reading syms from /usr/lib64/libstdc++.so.6.0.8 (0x3739400000)
>> --30004-- object doesn't have a symbol table
>> --30004-- Reading syms from /lib64/libm-2.5.so <http://libm-2.5.so>
>> (0x3735800000)
>>
>> --30004-- Reading syms from /lib64/libgcc_s-4.1.2-20080825.so.1
>> (0x3738C00000)
>> --30004-- object doesn't have a symbol table
>> --30004-- Reading syms from /lib64/libc-2.5.so <http://libc-2.5.so>
>> (0x3735000000)
>>
>> --30004-- REDIR: 0x3735079AB0 (rindex) redirected to 0x4A06400 (rindex)
>> --30004-- REDIR: 0x373507A970 (memset) redirected to 0x4A06920 (memset)
>> --30004-- REDIR: 0x37350796C0 (strlen) redirected to 0x4A066C0 (strlen)
>> --30004-- REDIR: 0x37394BD160 (operator new(unsigned long)) redirected
>> to 0x4A05F95 (operator new(unsigned long))
>> --30004-- REDIR: 0x373507BDB0 (memcpy) redirected to 0x4A06FF0 (memcpy)
>> ==30004== Conditional jump or move depends on uninitialised value(s)
>> ==30004== at 0x373949B2BD: (within /usr/lib64/libstdc++.so.6.0.8)
>> ==30004== by 0x373949B585: std::string::find(char const*, unsigned
>> long, unsigned long) const (in /usr/lib64/libstdc++.so.6.0.8)
>> ==30004== by 0x402621:
>> libXML::CParser::changeCharsToEntitiesInCDATA() (in
>> /home/kubinski/max/22/ala)
>> ==30004== by 0x404259: libXML::CParser::parse(std::string&) (in
>> /home/kubinski/max/22/ala)
>> ==30004== by 0x4017E3: main (in /home/kubinski/max/22/ala)
>> --30004-- REDIR: 0x37394BBEF0 (operator delete(void*)) redirected to
>> 0x4A050A9 (operator delete(void*))
>> --30004-- REDIR: 0x373507A1B0 (memchr) redirected to 0x4A06850 (memchr)
>> ==30004==
>> ==30004== Conditional jump or move depends on uninitialised value(s)
>> ==30004== at 0x404167: libXML::CParser::parse() (in
>> /home/kubinski/max/22/ala)
>> ==30004== by 0x404262: libXML::CParser::parse(std::string&) (in
>> /home/kubinski/max/22/ala)
>> ==30004== by 0x4017E3: main (in /home/kubinski/max/22/ala)
>> --30004-- REDIR: 0x37350726F0 (free) redirected to 0x4A05397 (free)
>> ==30004==
>> ==30004== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 1)
>> ==30004==
>> ==30004== 1 errors in context 1 of 2:
>> ==30004== Conditional jump or move depends on uninitialised value(s)
>> ==30004== at 0x404167: libXML::CParser::parse() (in
>> /home/kubinski/max/22/ala)
>> ==30004== by 0x404262: libXML::CParser::parse(std::string&) (in
>> /home/kubinski/max/22/ala)
>> ==30004== by 0x4017E3: main (in /home/kubinski/max/22/ala)
>> ==30004==
>> ==30004== 1 errors in context 2 of 2:
>> ==30004== Conditional jump or move depends on uninitialised value(s)
>> ==30004== at 0x373949B2BD: (within /usr/lib64/libstdc++.so.6.0.8)
>> --30004--
>> --30004-- supp: 4 Fedora-Core-6-hack3-ld25
>> ==30004==
>> ==30004== IN SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 1)
>> ==30004==
>> ==30004== malloc/free: in use at exit: 0 bytes in 0 blocks.
>> ==30004== malloc/free: 846,934 allocs, 846,934 frees, 25,055,370 bytes
>> allocated.
>> ==30004==
>> ==30004== All heap blocks were freed -- no leaks are possible.
>> --30004-- memcheck: sanity checks: 11546 cheap, 462 expensive
>> --30004-- memcheck: auxmaps: 179 auxmap entries (11456k, 11M) in use
>> --30004-- memcheck: auxmaps: 27975161 searches, 29272608 comparisons
>> --30004-- memcheck: SMs: n_issued = 481 (7696k, 7M)
>> --30004-- memcheck: SMs: n_deissued = 18 (288k, 0M)
>> --30004-- memcheck: SMs: max_noaccess = 524287 (8388592k, 8191M)
>> --30004-- memcheck: SMs: max_undefined = 9 (144k, 0M)
>> --30004-- memcheck: SMs: max_defined = 249 (3984k, 3M)
>> --30004-- memcheck: SMs: max_non_DSM = 481 (7696k, 7M)
>> --30004-- memcheck: max sec V bit nodes: 0 (0k, 0M)
>> --30004-- memcheck: set_sec_vbits8 calls: 0 (new: 0, updates: 0)
>> --30004-- memcheck: max shadow mem size: 11840k, 11M
>> --30004-- translate: fast SP updates identified: 3,741 ( 87.8%)
>> --30004-- translate: generic_known SP updates identified: 307 ( 7.2%)
>> --30004-- translate: generic_unknown SP updates identified: 209 ( 4.9%)
>> --30004-- tt/tc: 117,211 tt lookups requiring 186,235 probes
>> --30004-- tt/tc: 117,211 fast-cache updates, 5 flushes
>> --30004-- transtab: new 3,358 (81,817 -> 1,555,815; ratio
>> 190:10) [0 scs]
>> --30004-- transtab: dumped 0 (0 -> ??)
>> --30004-- transtab: discarded 12 (257 -> ??)
>> --30004-- scheduler: 1,154,615,962 jumps (bb entries).
>> --30004-- scheduler: 11,546/1,819,427 major/minor sched events.
>> --30004-- sanity: 11547 cheap, 462 expensive checks.
>> --30004-- exectx: 30,011 lists, 170 contexts (avg 0 per list)
>> --30004-- exectx: 1,693,874 searches, 21,706,760 full compares
>> (12,814 per 1000)
>> --30004-- exectx: 0 cmp2, 15 cmp4, 0 cmpAll
>>
>>
>>
>> proc status without the valgrind after the XML creation
>> Name: ala
>> State: S (sleeping)
>> Tgid: 889
>> Pid: 889
>> PPid: 27692
>> TracerPid: 0
>> Uid: 61368538 61368538 61368538 61368538
>> Gid: 300 300 300 300
>> FDSize: 256
>> Groups: 300 2051
>> VmPeak: 21724 kB
>> VmSize: 21724 kB
>> VmLck: 0 kB
>> VmHWM: 11048 kB
>> VmRSS: 11048 kB
>> VmData: 9700 kB
>> VmStk: 136 kB
>> VmExe: 688 kB
>> VmLib: 2936 kB
>> VmPTE: 76 kB
>> VmSwap: 0 kB
>> Threads: 1
>> SigQ: 0/1033564
>> SigPnd: 0000000000000000
>> ShdPnd: 0000000000000000
>> SigBlk: 0000000000000000
>> SigIgn: 0000000000000000
>> SigCgt: 0000000000000000
>> CapInh: 0000000000000000
>> CapPrm: 0000000000000000
>> CapEff: 0000000000000000
>> CapBnd: ffffffffffffffff
>> Cpus_allowed: ffffffff
>> Cpus_allowed_list: 0-31
>> voluntary_ctxt_switches: 13
>> nonvoluntary_ctxt_switches: 2
>>
>> proc status without the valgrind after deleting the XML node:
>> Name: ala
>> State: S (sleeping)
>> Tgid: 889
>> Pid: 889
>> PPid: 27692
>> TracerPid: 0
>> Uid: 61368538 61368538 61368538 61368538
>> Gid: 300 300 300 300
>> FDSize: 256
>> Groups: 300 2051
>> VmPeak: 21724 kB
>> VmSize: 21724 kB
>> VmLck: 0 kB
>> VmHWM: 11184 kB
>> VmRSS: 11184 kB
>> VmData: 9700 kB
>> VmStk: 136 kB
>> VmExe: 688 kB
>> VmLib: 2936 kB
>> VmPTE: 76 kB
>> VmSwap: 0 kB
>> Threads: 1
>> SigQ: 0/1033564
>> SigPnd: 0000000000000000
>> ShdPnd: 0000000000000000
>> SigBlk: 0000000000000000
>> SigIgn: 0000000000000000
>> SigCgt: 0000000000000000
>> CapInh: 0000000000000000
>> CapPrm: 0000000000000000
>> CapEff: 0000000000000000
>> CapBnd: ffffffffffffffff
>> Cpus_allowed: ffffffff
>> Cpus_allowed_list: 0-31
>> voluntary_ctxt_switches: 14
>> nonvoluntary_ctxt_switches: 4
>>
>> Br,
>> Jakub
>>
>>
>> ------------------------------------------------------------------------------
>> Keep Your Developer Skills Current with LearnDevNow!
>> The most comprehensive online learning library for Microsoft developers
>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> Metro Style Apps, more. Free future releases when you subscribe now!
>> http://p.sf.net/sfu/learndevnow-d2d
>> _______________________________________________
>> Valgrind-users mailing list
>> [email protected]
>> <mailto:[email protected]>
>> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Keep Your Developer Skills Current with LearnDevNow!
>> The most comprehensive online learning library for Microsoft developers
>> is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3,
>> Metro Style Apps, more. Free future releases when you subscribe now!
>> http://p.sf.net/sfu/learndevnow-d2d
>>
>>
>>
>> _______________________________________________
>> Valgrind-users mailing list
>> [email protected]
>> https://lists.sourceforge.net/lists/listinfo/valgrind-users
>
>
>
> --
> -- Howard Chu
> CTO, Symas Corp. http://www.symas.com
> Director, Highland Sun http://highlandsun.com/hyc/
> Chief Architect, OpenLDAP http://www.openldap.org/project/
------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Valgrind-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/valgrind-users
