Sorry I've got no insight on this.
On Apr 21, 2016 6:29 AM, "Biddiscombe, John A." <[email protected]> wrote:
Dave
So it turns out that aprun/alps won’t let me run a script with dstat to
monitor the memory use, and using craypat makes the execution take too long
(weeks it seems). So I tried importing part of the dstat python into my
paraview python
Using this I am able to query memory use on the node and dump it out.
Question : If I start a background python thread that polls memory like
this
#####################
# start a thread that prints mem use
#####################
def dump_memory():
# do something here ...
global stop_thread
line = ''
for o in stats:
o.extract()
line = line + ' ' + o.show()
print line + dstat.ansi['reset']
# call f() again in n seconds
if stop_thread==0:
threading.Timer(0.5, dump_memory).start()
and set it going at the start of my paraview python pipeline : Will it
actually run and collect info whilst the paraview script is executing the
rest of the pipeline.
intuition and brief googling tells me that this won’t work.
Can it be done?
JB
*From:* ParaView [mailto:[email protected]] *On Behalf Of
*Biddiscombe,
John A.
*Sent:* 18 April 2016 14:53
*To:* David E DeMarle
*Cc:* [email protected]
*Subject:* Re: [Paraview] peak memory use on remote nodes
Thanks Dave
A colleague here suggested I put my
aprun pvbatch <blah> into a script and add a ‘dtrace” command at the start
of the script. This will then dump out a txt file on each node and after
the job completes I can grep the info I need.
I’m going to give that a try first as it seems straightforward, but I will
revert to your suggestion if I fail and nothing else comes up.
cheers
JB
*From:* David E DeMarle [mailto:[email protected]
<[email protected]>]
*Sent:* 18 April 2016 14:48
*To:* Biddiscombe, John A.
*Cc:* [email protected]
*Subject:* Re: [Paraview] peak memory use on remote nodes
Perhaps extend the benchmark to glean /proc/<PID>/status's VmHWM entry on
Linux at least. Get the number for t1 and at t2 and subtract to see what
the maximum was in between those two times.
Seems the right place to do that is kwsys/SystemInformation.cxx, but you
could prototype in the script.
Hope that helps and please post some code. Seems like a very useful thing
to have.
David E DeMarle
Kitware, Inc.
R&D Engineer
21 Corporate Drive
Clifton Park, NY 12065-8662
Phone: 518-881-4909
On Mon, Apr 18, 2016 at 2:42 AM, Biddiscombe, John A. <[email protected]>
wrote:
The python scripting module “paraview.benchmark” allows one to get the
memory use using “paraview.benchmark.get_memuse” - but I presume this is
the memory reported as used by the system at the moment when the function
is called.
Does anyone know a way of recording the peak memory use on remote nodes
between t1 and t2 - where t1 and t2 are the start and stop of either a
function, job or even paraview python script on the remote node?
thanks
JB
--
John Biddiscombe, email:biddisco @.at.@ cscs.ch
http://www.cscs.ch/
CSCS, Swiss National Supercomputing Centre | Tel: +41 (91) 610.82.07
Via Trevano 131, 6900 Lugano, Switzerland | Fax: +41 (91) 610.82.82
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Search the list archives at: http://markmail.org/search/?q=ParaView
Follow this link to subscribe/unsubscribe:
http://public.kitware.com/mailman/listinfo/paraview
