I guess I sort of answered my own question -- the entire script runs on each
processor, so I ended up with 8 copies of my data in memory (or I would have,
had I not filled the 12 GB of RAM and 20 GB of swap space and my system
crashed).
So is there some way to query the processor information? Probably something in
the RequestInformation script -- find out how many processors there are and
then the prog. filter determines based on processor ID and number of processors
what section of the data to load.
In that case, how does the aggregation of the data work? The exact pipeline is:
DataObjectGenerator("MB{}")
ProgrammableFilter
in serial, the PF appends blocks into the input and passes that through to the
output. In parallel, that same pipeline would create a MB{} on each CPU that
gets filled with that CPU's data, but at the end of this step I would want a
single MB{} object, not NCPU MB{}'s.
Hopefully that makes sense... I've never used PV in parallel, so I'm not sure
how it all works.
Tim
----- Original Message -----
From: "Tim Gallagher" <[email protected]>
To: "ParaView list" <[email protected]>
Sent: Wednesday, August 3, 2011 9:24:25 AM
Subject: [Paraview] Programmable filter in parallel
Hi,
I know many of the built-in readers/filters already work in parallel, but how
does one write a parallel programmable filter?
Our data files are XDMF and split into blocks of data. We have a single XDMF
file that we can read that reads all the blocks and generates a
vtkMultiBlockDataset (this works with the built in XDMF reader).
However, each block has some ghost cells around it that are needed to do the
CellDataToPointData interpolation. For large numbers of blocks, this creates
far too many grid points for our machines to load. So, I've written a
programmable filter that does:
start with empty vtkMultiBlockDataset
for each block in restart file
read block file with XDMFReader
CellDataToPointData
strip off the extra layers of cells
append to output vtkMultiBlockDataset
If I run this in parallel, what exactly is parallel? Is the reading and CD2PD
done in parallel on each block? Is none of it parallel? Ideally, I would have
the loop over blocks done in parallel, but I don't know how to indicate that in
the programmable filter (if it's possible).
Any advice would be great,
Tim
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
