Hi,
If you know the structure of the binary file, you can use xdmf format
file to read your binary data.
for the binary part use something like this:
<DataItem ItemType="Uniform"
Format="Binary"
Dimensions="64 128 256"
Seek="1234">
PressureFile.bin
</DataItem>
if you grid does not change over time you can use XPath:
"XPath, allows for elements in the XML document and the API to
reference specific elements in a document"
http://www.xdmf.org/index.php/XDMF_Model_and_Format
I use xdmf all the time, I think is very flexible (ascii, binary, HDF5
heavy data storage), time, grids, extra data using "Information" tag.
And all kinds of topologies and geometries.
Le 20 mai 12 à 13:47, Hal Canary a écrit :
On 05/19/2012 06:30 PM, Mohammad Mirzadeh wrote:
Hi guys,
I need a little bit of advice on how I'm outputting data. So far I
have
been using a simple legacy VTK format and dumping my grid and all
sclaar/vectors into that single file. Basically I start with
writing the
grid and then append as many as variables that I like to the file.
This
has the problem that it was ascii and slow for large data sets,
especially fro a parallel code where the solver was taking only 5 sec
but the IO was around 20!!
So I want to turn into binary and I have learned how to output binary
vtu files. The issue is, since the cluster I'm working with has kill
times (Max 24 hr) I also need to be able to save and resume my
simulation. So, what I do right now is dump my variables in binary
form
into a single file that does not have any xml tag or anything -- just
raw binary data. This is a preferred method since when I want to
resume,
I do not need to parse the xml code; I just read the binary file and
that's it. Besides, I save on the grid information I'm writing to
disk.
For a 23M grid point, the grid information (which basically is a
quadtree saved as a general unstructured grid with
cell connectivity information etc) file is about 1GB whereas my
actual
data is only about 150MB. In most cases my grid does not change with
time, so I really only need to save it once and that's enough.
Now, the problem is, I want to be able to use or refer to these
binary
files using the xml tags. Here are the questions:
1) Is this supported in .vtu format? How can I refer to a binary file
instead of dumping all the binary information in the file?
2) If this is not supported in .vtu, is it something xdmf can do?
If so
how? Can you please point me to a tutorial?
3) Is there any plan or strategy that would be better than what I
have
in mind right now? Please remember that retaining the raw binary
format
of the files is a must for me (well I do not see convincing reasons
to
give up on it, but if there is one, please let me know)
Thanks
Mohammad
Just this week, I wrote a Python program that uses the
numpy.fromfile function and ParaView's
paraview.vtk.dataset_adapter.numpyTovtkDataArray and
paraview.vtk.io.vtkXMLPolyDataWriter functions to convert a large
binary file to .vtp format. It is reasonably fast. Maybe you can
use something like that as a post-processing step.
This separates the tasks of producing data and converting it to VTK
format.
--
Hal Canary
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
