Hi Philippe,
Thank you for your reply.
>Ok, the origin of the memory is related to the locks/nr of locks
>your application creates and/or locks/keeps locked.
>About 50Gb of memory is consumed for hg.ids.4.
What is the hg.ids.4? Is it helgrind's memory using for analysing lock?
I know that a lock can not cost big memory, how many memory does helgrind cost
for analysing one lock?
It is unbelievable that helgrind will cost 50GB for 1,029,288 locks.
>From the above, I guess your application has thousands
>of locks, and that each thread has sometimes hundreds
>of locks held.Myprog have creates about 4096*64 locks, sometimes myprog will
>lock almost all the locks at one time, and release them later.
>There is no garbage collection of the lock set universe.
You mean the big memory allocated using for analysing locks can not be
reused/recyled by valgrind?
>At this point, I think there are 2 possible approaches:
> * implement in helgrind a garbage collection of the univ_lsets
> (unclear if this is implementable, and for sure not trivial)
> * you change the limits of Valgrind max memory, to e.g. go
> to 128G or 256G.
Both ways are so difficult for me, I am want to try the second way, but I need
your your help.
Can you give me a patch for 3.10.1 to change the limits of Valgrind max memory
to 256G.Thanks.
> I have done it two times with different options series. The result is
> as follows:
> 1.--tool=helgrind --gen-suppressions=all --stats=yes
> --profile-heap=yes --track-lockorders=no --read-var-info=yes
> --trace-children=yes --error-limit=no --log-file=xxx.log myprog
Ok, the origin of the memory is related to the locks/nr of locks
your application creates and/or locks/keeps locked.
About 50Gb of memory is consumed for hg.ids.4.
> 51,760,182,272 in 341,498: hg.ids.4
This is the 'universe of locksets' : ie. all the locksets that
helgrind has ever seen.
It looks like your application uses a lot of locks
and keeps a lot of locks in a status 'locked' shown by:
> locks: 1,029,288 acquires, 915,546 releases
From the above, I guess your application has thousands
of locks, and that each thread has sometimes hundreds
of locks held.
Well, if that is the case, I think helgrind data structure
is not done for such a behaviour :(
There is no garbage collection of the lock set universe.
At this point, I think there are 2 possible approaches:
* implement in helgrind a garbage collection of the univ_lsets
(unclear if this is implementable, and for sure not trivial)
* you change the limits of Valgrind max memory, to e.g. go
to 128G or 256G.
You might refer for that to the revision r13278, which increased
from 32G to 64G. That gives the various constants to multiply by 2
or 4 to go to 128G or 256G.
So, checkout the SVN version, then do
svn diff -r13277:13278
and follow that pattern to increase to 128 or 256G
Philippe------------------------------------------------------------------------------
_______________________________________________
Valgrind-users mailing list
Valgrind-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/valgrind-users
