Jakub Kubinski wrote:
> Hi,
>
> I tried tcmalloc but the results are the same. RSS memory usage is
> even slightly higher when I used tcmalloc
This test is inconclusive since tcmalloc never returns freed memory to the OS.
Also, while tcmalloc has greater overhead in a single-threaded program, it has
much less overhead than glibc in multi-threaded code.
>
> RSS after main initialization
> glibcMalloc : 2080KB
> tcMalloc: 2756KB
>
> RSS after XML node initialization
> glibcMalloc : 7004KB
> tcMalloc: 7560KB
>
> RSS after removing the XML node
> glibcMalloc : 7068KB
> tcMalloc: 7764KB
>
>
> I also created a small test program in which I manage to reproduce the
> same results:
>
> #include<stdlib.h>
> #include<stdio.h>
> #include<string>
> using namespace std;
>
> int main()
> {
> system("top -b -n 1> result_top.txt");
> int **t = new int*[100000];
> for(long i = 0; i< 100000; i++)
> {
> t[i] = new int();
>
> }
> system("top -b -n 1>> result_top.txt");
>
> for(long i = 100000 - 1; i>= 0; i--)
> {
> delete t[i];
> }
> delete [] t;
> system("top -b -n 1>> result_top.txt");
> return 0;
> }
>
> Top output after main initialization:
> glibc: 2716
> tcMalloc: 4948
>
> Top output after âtâ array initialization:
> glibc: 2572
> tcMalloc: 2668
>
> Top output after deleting the âtâ array:
> glibc: 2352
> tcMalloc: 2708
>
> Br,
> Jakub
>
>
>
> On 14 February 2012 20:20, Howard Chu<h...@symas.com> wrote:
>> John Alvord wrote:
>>>
>>> I am an interested bystander. Last year I diagnosed a 2+ year long storage
>>> growth issue where there was no storage leak... firmly proven.
>>>
>>> The diagnosis was a case of extreme fragmentation. The process needed
>>> large
>>> chunks of contiguous storage [2-3 megs] and when released the system
>>> storage
>>> allocator [AIX] did not segregate the large chunks. Smaller requests
>>> populated
>>> the newly freed area "sometimes" and thus a new storage chunk was needed.
>>> When
>>> there was no storage available for a large chunk, the process space was
>>> increased in a large chunk. Repeat until the upper limit of a 32 bit AIX
>>> process size [roughly 2 gigs] is exceeded and the process failed, which
>>> took
>>> 10-12 days.
>>>
>>> Not sure how you could prove that case - in effect would need to look at
>>> the
>>> physical layout of allocated and free storage.
>>
>>
>> We see this all the time in OpenLDAP slapd with the glibc allocator. I've
>> documented it pretty extensively. Google tcmalloc is much less susceptible
>> to the problem.
>>
>> http://highlandsun.com/hyc/malloc/
>>
>>>
>>> Just one possibility...
>>>
>>> John Alvord, IBM
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: *Jakub Kubinski*<jkubinsk...@gmail.com
>>> <mailto:jkubinsk...@gmail.com>>
>>> Date: Tue, Feb 14, 2012 at 3:23 AM
>>> Subject: [Valgrind-users] RSS memory is increasing but memory leaks are
>>> not
>>> reported
>>> To: valgrind-users@lists.sourceforge.net
>>> <mailto:valgrind-users@lists.sourceforge.net>
>>>
>>>
>>> Hello,
>>>
>>> I have a "strange" memory leak in my XML parser which I cannot locate
>>> using valgrind.
>>>
>>> In my testcase I'm creating and deleting a XML node and after the test
>>> valgrind is not showing any memory leaks but rss memory is still
>>> active VmRSS: 11184 kB ( VmRSS after the XML node creation was
>>> 11048 kB so it increased by 136 after executing the XML node
>>> destructor )
>>>
>>> In the valigirng report I found "--30004-- memcheck: auxmaps: 179
>>> auxmap entries (11456k, 11M) in use" which fits to the remaining 11048
>>> kB in the RSS memory.What does this mean ?
>>>
>>>
>>>
>>> ==30004== Memcheck, a memory error detector.
>>> ==30004== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al.
>>> ==30004== Using LibVEX rev 1658, a library for dynamic binary translation.
>>> ==30004== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP.
>>> ==30004== Using valgrind-3.2.1, a dynamic binary instrumentation
>>> framework.
>>> ==30004== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al.
>>> ==30004==
>>> ==30004== My PID = 30004, parent PID = 27692. Prog and args are:
>>> ==30004== ./ala
>>> ==30004==
>>> --30004--
>>> --30004-- Command line
>>> --30004-- ./ala
>>> --30004-- Startup, with flags:
>>> --30004-- --tool=memcheck
>>> --30004-- --leak-check=full
>>> --30004-- --leak-resolution=high
>>> --30004-- --num-callers=40
>>> --30004-- --show-reachable=yes
>>> --30004-- --error-limit=no
>>> --30004-- -v
>>> --30004-- --log-file=valgrind.log
>>> --30004-- Contents of /proc/version:
>>> --30004-- Linux version 2.6.39.4-1.NSN.kiuas
>>> (r...@trling23.emea.nsn-net.net<mailto:r...@trling23.emea.nsn-net.net>)
>>> (gcc
>>>
>>> version 4.1.2 20080704 (Red Hat
>>> 4.1.2-46)) #1 SMP Wed Aug 10 12:16:36 EEST 2011
>>> --30004-- Arch and hwcaps: AMD64, amd64-sse2
>>> --30004-- Valgrind library directory: /usr/lib64/valgrind
>>> --30004-- Reading syms from /home/kubinski/max/22/ala (0x400000)
>>> --30004-- Reading syms from /usr/lib64/valgrind/amd64-linux/memcheck
>>> (0x38000000)
>>> --30004-- object doesn't have a dynamic symbol table
>>> --30004-- Reading syms from /lib64/ld-2.5.so<http://ld-2.5.so>
>>> (0x3734000000)
>>>
>>> --30004-- Reading suppressions file: /usr/lib64/valgrind/default.supp
>>> --30004-- Reading syms from
>>> /usr/lib64/valgrind/amd64-linux/vgpreload_core.so (0x4802000)
>>> --30004-- Reading syms from
>>> /usr/lib64/valgrind/amd64-linux/vgpreload_memcheck.so (0x4A03000)
>>> --30004-- REDIR: 0x3734014400 (index) redirected to 0x4A06550 (index)
>>> --30004-- REDIR: 0x37340145B0 (strcmp) redirected to 0x4A067D0 (strcmp)
>>> --30004-- REDIR: 0x37340145E0 (strlen) redirected to 0x4A06700 (strlen)
>>> --30004-- Reading syms from /usr/lib64/libstdc++.so.6.0.8 (0x3739400000)
>>> --30004-- object doesn't have a symbol table
>>> --30004-- Reading syms from /lib64/libm-2.5.so<http://libm-2.5.so>
>>> (0x3735800000)
>>>
>>> --30004-- Reading syms from /lib64/libgcc_s-4.1.2-20080825.so.1
>>> (0x3738C00000)
>>> --30004-- object doesn't have a symbol table
>>> --30004-- Reading syms from /lib64/libc-2.5.so<http://libc-2.5.so>
>>> (0x3735000000)
>>>
>>> --30004-- REDIR: 0x3735079AB0 (rindex) redirected to 0x4A06400 (rindex)
>>> --30004-- REDIR: 0x373507A970 (memset) redirected to 0x4A06920 (memset)
>>> --30004-- REDIR: 0x37350796C0 (strlen) redirected to 0x4A066C0 (strlen)
>>> --30004-- REDIR: 0x37394BD160 (operator new(unsigned long)) redirected
>>> to 0x4A05F95 (operator new(unsigned long))
>>> --30004-- REDIR: 0x373507BDB0 (memcpy) redirected to 0x4A06FF0 (memcpy)
>>> ==30004== Conditional jump or move depends on uninitialised value(s)
>>> ==30004== at 0x373949B2BD: (within /usr/lib64/libstdc++.so.6.0.8)
>>> ==30004== by 0x373949B585: std::string::find(char const*, unsigned
>>> long, unsigned long) const (in /usr/lib64/libstdc++.so.6.0.8)
>>> ==30004== by 0x402621:
>>> libXML::CParser::changeCharsToEntitiesInCDATA() (in
>>> /home/kubinski/max/22/ala)
>>> ==30004== by 0x404259: libXML::CParser::parse(std::string&) (in
>>> /home/kubinski/max/22/ala)
>>> ==30004== by 0x4017E3: main (in /home/kubinski/max/22/ala)
>>> --30004-- REDIR: 0x37394BBEF0 (operator delete(void*)) redirected to
>>> 0x4A050A9 (operator delete(void*))
>>> --30004-- REDIR: 0x373507A1B0 (memchr) redirected to 0x4A06850 (memchr)
>>> ==30004==
>>> ==30004== Conditional jump or move depends on uninitialised value(s)
>>> ==30004== at 0x404167: libXML::CParser::parse() (in
>>> /home/kubinski/max/22/ala)
>>> ==30004== by 0x404262: libXML::CParser::parse(std::string&) (in
>>> /home/kubinski/max/22/ala)
>>> ==30004== by 0x4017E3: main (in /home/kubinski/max/22/ala)
>>> --30004-- REDIR: 0x37350726F0 (free) redirected to 0x4A05397 (free)
>>> ==30004==
>>> ==30004== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 1)
>>> ==30004==
>>> ==30004== 1 errors in context 1 of 2:
>>> ==30004== Conditional jump or move depends on uninitialised value(s)
>>> ==30004== at 0x404167: libXML::CParser::parse() (in
>>> /home/kubinski/max/22/ala)
>>> ==30004== by 0x404262: libXML::CParser::parse(std::string&) (in
>>> /home/kubinski/max/22/ala)
>>> ==30004== by 0x4017E3: main (in /home/kubinski/max/22/ala)
>>> ==30004==
>>> ==30004== 1 errors in context 2 of 2:
>>> ==30004== Conditional jump or move depends on uninitialised value(s)
>>> ==30004== at 0x373949B2BD: (within /usr/lib64/libstdc++.so.6.0.8)
>>> --30004--
>>> --30004-- supp: 4 Fedora-Core-6-hack3-ld25
>>> ==30004==
>>> ==30004== IN SUMMARY: 2 errors from 2 contexts (suppressed: 4 from 1)
>>> ==30004==
>>> ==30004== malloc/free: in use at exit: 0 bytes in 0 blocks.
>>> ==30004== malloc/free: 846,934 allocs, 846,934 frees, 25,055,370 bytes
>>> allocated.
>>> ==30004==
>>> ==30004== All heap blocks were freed -- no leaks are possible.
>>> --30004-- memcheck: sanity checks: 11546 cheap, 462 expensive
>>> --30004-- memcheck: auxmaps: 179 auxmap entries (11456k, 11M) in use
>>> --30004-- memcheck: auxmaps: 27975161 searches, 29272608 comparisons
>>> --30004-- memcheck: SMs: n_issued = 481 (7696k, 7M)
>>> --30004-- memcheck: SMs: n_deissued = 18 (288k, 0M)
>>> --30004-- memcheck: SMs: max_noaccess = 524287 (8388592k, 8191M)
>>> --30004-- memcheck: SMs: max_undefined = 9 (144k, 0M)
>>> --30004-- memcheck: SMs: max_defined = 249 (3984k, 3M)
>>> --30004-- memcheck: SMs: max_non_DSM = 481 (7696k, 7M)
>>> --30004-- memcheck: max sec V bit nodes: 0 (0k, 0M)
>>> --30004-- memcheck: set_sec_vbits8 calls: 0 (new: 0, updates: 0)
>>> --30004-- memcheck: max shadow mem size: 11840k, 11M
>>> --30004-- translate: fast SP updates identified: 3,741 ( 87.8%)
>>> --30004-- translate: generic_known SP updates identified: 307 ( 7.2%)
>>> --30004-- translate: generic_unknown SP updates identified: 209 ( 4.9%)
>>> --30004-- tt/tc: 117,211 tt lookups requiring 186,235 probes
>>> --30004-- tt/tc: 117,211 fast-cache updates, 5 flushes
>>> --30004-- transtab: new 3,358 (81,817 -> 1,555,815; ratio
>>> 190:10) [0 scs]
>>> --30004-- transtab: dumped 0 (0 -> ??)
>>> --30004-- transtab: discarded 12 (257 -> ??)
>>> --30004-- scheduler: 1,154,615,962 jumps (bb entries).
>>> --30004-- scheduler: 11,546/1,819,427 major/minor sched events.
>>> --30004-- sanity: 11547 cheap, 462 expensive checks.
>>> --30004-- exectx: 30,011 lists, 170 contexts (avg 0 per list)
>>> --30004-- exectx: 1,693,874 searches, 21,706,760 full compares
>>> (12,814 per 1000)
>>> --30004-- exectx: 0 cmp2, 15 cmp4, 0 cmpAll
>>>
>>>
>>>
>>> proc status without the valgrind after the XML creation
>>> Name: ala
>>> State: S (sleeping)
>>> Tgid: 889
>>> Pid: 889
>>> PPid: 27692
>>> TracerPid: 0
>>> Uid: 61368538 61368538 61368538 61368538
>>> Gid: 300 300 300 300
>>> FDSize: 256
>>> Groups: 300 2051
>>> VmPeak: 21724 kB
>>> VmSize: 21724 kB
>>> VmLck: 0 kB
>>> VmHWM: 11048 kB
>>> VmRSS: 11048 kB
>>> VmData: 9700 kB
>>> VmStk: 136 kB
>>> VmExe: 688 kB
>>> VmLib: 2936 kB
>>> VmPTE: 76 kB
>>> VmSwap: 0 kB
>>> Threads: 1
>>> SigQ: 0/1033564
>>> SigPnd: 0000000000000000
>>> ShdPnd: 0000000000000000
>>> SigBlk: 0000000000000000
>>> SigIgn: 0000000000000000
>>> SigCgt: 0000000000000000
>>> CapInh: 0000000000000000
>>> CapPrm: 0000000000000000
>>> CapEff: 0000000000000000
>>> CapBnd: ffffffffffffffff
>>> Cpus_allowed: ffffffff
>>> Cpus_allowed_list: 0-31
>>> voluntary_ctxt_switches: 13
>>> nonvoluntary_ctxt_switches: 2
>>>
>>> proc status without the valgrind after deleting the XML node:
>>> Name: ala
>>> State: S (sleeping)
>>> Tgid: 889
>>> Pid: 889
>>> PPid: 27692
>>> TracerPid: 0
>>> Uid: 61368538 61368538 61368538 61368538
>>> Gid: 300 300 300 300
>>> FDSize: 256
>>> Groups: 300 2051
>>> VmPeak: 21724 kB
>>> VmSize: 21724 kB
>>> VmLck: 0 kB
>>> VmHWM: 11184 kB
>>> VmRSS: 11184 kB
>>> VmData: 9700 kB
>>> VmStk: 136 kB
>>> VmExe: 688 kB
>>> VmLib: 2936 kB
>>> VmPTE: 76 kB
>>> VmSwap: 0 kB
>>> Threads: 1
>>> SigQ: 0/1033564
>>> SigPnd: 0000000000000000
>>> ShdPnd: 0000000000000000
>>> SigBlk: 0000000000000000
>>> SigIgn: 0000000000000000
>>> SigCgt: 0000000000000000
>>> CapInh: 0000000000000000
>>> CapPrm: 0000000000000000
>>> CapEff: 0000000000000000
>>> CapBnd: ffffffffffffffff
>>> Cpus_allowed: ffffffff
>>> Cpus_allowed_list: 0-31
>>> voluntary_ctxt_switches: 14
>>> nonvoluntary_ctxt_switches: 4
>>>
>>> Br,
>>> Jakub
--
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
------------------------------------------------------------------------------
Virtualization & Cloud Management Using Capacity Planning
Cloud computing makes use of virtualization - but cloud computing
also focuses on allowing computing to be delivered as a service.
http://www.accelacomm.com/jaw/sfnl/114/51521223/
_______________________________________________
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
