KuN,
Normally as datasets get larger you'd want to use multiple nodes each
with multiple cores. 7GB unstructured data is maybe big for one node. I
think a 64GB node should be able to handle it, although you're in
territory where you may want more nodes depending on what you're doing
in PV.
One issue may arise from the reader and how the data is stored on disk.
The legacy VTK format is the worst, the XML format can be very good
although with the XML formats I think it works best when the data is
partitioned into chunks on disk. Which format are you using and if it's
the XML format how many partitions on disk are there?
A related issue is that MPI used int's in its API. This sometimes is a
problem when dealing with really large data. This won't be an issue if
your data is in the XML format and partitioned into 2G or smaller chunks.
In PV 4.1 the command line option --enable-bt will produce a stack trace
on stderr if PV crashes or aborts, you'd need to compile with debug
symbols, CMAKE_BUILD_TYPE=Debug or RelWithDebInfo. This may shed light
on what's crashing and why. In earlier versions backtrace could be
enabled from the memory inspector panel but didn't catch abort and will
not help for debugging MPI errors.
Burlen
On 12/13/2013 03:50 AM, KuN wrote:
Hi all,
I've been using PV for a while. Just recently I start to feed
hi-resoultion LiDAR to my model. The output file is huge, 7gb with 23
million node and even more triangles. Of course my workstation can't
render this kind of data (I process them on a Amazon EC2 with 64GB
ram), so I opened another EC2 and installed a new PV with MPI&OSMESA
enabled. The remote server works well with small files. But it crashed
with bigger one while multi-cores were used.
"BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES"
"EXIT CODE :134"
It did work with one core only, but the speed is unbearable. I also
noticed that when it worked (with one core), it only used about 11.1
GB of ram while the release ver. used 13.6GB in my Ubuntu 12.04 before
crashing.
Sorry I can't provide anymore useful information. But I'd like to know
how you guys work with this kind of big VTK file.
Thanks and have a good weekend,
KuN
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
