Takuya, thanks for your quick reply. however, we do have set
VTK_USE_64BIT_IDS ON and this is a 64bit linux platform (CentOS/RHEL x86_64) in case that matters - so unfortunately that isn't the solution here. thanks anyway and kind regards, guenter On Tue, Jan 4, 2011 at 12:06 PM, Takuya OSHIMA <[email protected]> wrote: > Hi Guenter and all, > > I also had the same issue with .vti (XML vtkImageData) files of > roughly the same sizes of around 3 GBytes. Currently I'm working > around the issue by building VTK and ParaView with VTK_USE_64BIT_IDS > set to ON, which will allow you to load up to 4 GBytes as compared to > 2 GBytes when VTK_USE_64BIT_IDS is OFF. > > However I'd be interested if there's more fundamental way of resolving > the issue because I'll be handling larger cases soon. Is there a > format (other than series of legacy VTK format files) that is > time-aware, parallelizable and allows more than 4 GBytes of > vtkImageData per partition? > > Takuya > > Takuya OSHIMA, Ph.D. > Faculty of Engineering, Niigata University > 8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN > > From: Guenter Bartsch <[email protected]> > Subject: [Paraview] Error when reading large XMLUnstructuredData file > Date: Tue, 4 Jan 2011 11:45:18 +0100 > >> hi all, >> >> we have run into a problem reading a pretty large (3.6 GB) .vtu file. >> the file was produced by a fortran program that writes vtk data in the >> old ascii format which results in a 17G ascii file which takes very >> long to load in paraview (and results in an error message). so, we >> have written a converter program that reads the ascii file and outputs >> xml: >> >> >> int main(int argc, char *argp[]) >> { >> if (argc != 2) { >> cerr << "usage: " << argp[0] << " <vtk filename>\n\n"; >> return 1; >> } >> >> char *filename = argp[1]; >> >> cout << "Input filename: " << filename << "\n"; >> >> vtkSmartPointer<vtkUnstructuredGridReader> reader = >> vtkSmartPointer<vtkUnstructuredGridReader>::New(); >> reader->SetFileName(filename); >> >> char *ofilename = "foo.vtu"; >> cout << "Output filename: " << ofilename << "\n"; >> vtkSmartPointer<vtkXMLUnstructuredGridWriter> writer = >> vtkSmartPointer<vtkXMLUnstructuredGridWriter>::New(); >> writer->SetFileName(ofilename); >> writer->SetInputConnection(reader->GetOutputPort()); >> writer->Write(); >> >> return EXIT_SUCCESS; >> } >> >> this program runs fine. now, if i open the resulting file in paraview, >> i run into this error as soon as i hit apply: >> >> ERROR: In >> /opt/paraview-3.8.1/src/ParaView-3.8.1/VTK/IO/vtkXMLUnstructuredDataReader.cxx, >> line 652 >> vtkXMLUnstructuredGridReader (0x143f0c40): Cannot read cell >> connectivity from Cells in piece 0 because the "connectivity" array is >> not long enough. >> >> the converter did work on a smaller test data set and paraview could >> open it - only now with the large data set paraview fails. here is the >> header of the failing file: >> >> <?xml version="1.0"?> >> <VTKFile type="UnstructuredGrid" version="0.1" >> byte_order="LittleEndian" compressor="vtkZLibDataCompressor"> >> <UnstructuredGrid> >> <Piece NumberOfPoints="106496000" >> NumberOfCells="105122745" > >> <PointData Vectors="vectors"> >> <DataArray type="Float32" Name="vectors" >> NumberOfComponents="3" format="appended" RangeMin="0.0021820231521" >> RangeMax="24.459394872" offset="0" /> >> </PointData> >> <CellData> >> </CellData> >> <Points> >> <DataArray type="Float32" Name="Points" NumberOfComponents="3" >> format="appended" RangeMin="0" >> RangeMax="5007.1818403" offset="1580185620" /> >> </Points> >> <Cells> >> <DataArray type="Int64" Name="connectivity" format="appended" >> RangeMin="" RangeMax="" >> offset="2045521660" /> >> <DataArray type="Int64" Name="offsets" format="appended" >> RangeMin="" RangeMax="" >> offset="3504937336" /> >> <DataArray type="UInt8" Name="types" format="appended" >> RangeMin="" RangeMax="" >> offset="3671971228" /> >> </Cells> >> </Piece> >> </UnstructuredGrid> >> <AppendedData encoding="base64"> >> _WJgAAACAAAAAAAAAvXYAALF2AADJdgAAUHcAA... >> >> does anyone know what could be going on here? do we need to configure >> paraview to handle larger files? is this a bug? could the original >> ascii data set have been corrupt and the error reflects now in the xml >> file? >> >> thanks in advance and best regards, >> >> guenter >> _______________________________________________ >> Powered by www.kitware.com >> >> Visit other Kitware open-source projects at >> http://www.kitware.com/opensource/opensource.html >> >> Please keep messages on-topic and check the ParaView Wiki at: >> http://paraview.org/Wiki/ParaView >> >> Follow this link to subscribe/unsubscribe: >> http://www.paraview.org/mailman/listinfo/paraview > _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
