Takuya, On Tue, Jan 4, 2011 at 12:21 PM, Takuya OSHIMA <[email protected]> wrote: > Hi Guenter, just for confirmation, did you build both VTK that your > converter uses and ParaView with VTK_USE_64BIT_IDS ON?
yes - at least that's what i thought i did ;) . actually we have built paraview 3.8.1 from source which came with a statically embedded version of VTK. i have then built my converter against that VTK that came with paraview. i have blogged about my little converter program: https://sites.google.com/site/guenterbartsch/blog/myfirstvtkprogram so others may benefit from it - but i don't think i have any additional information in my blog post. thanks again for your help and best regards, guenter > > By the way I'm working with OS X 10.6, if it makes difference... > > Takuya > > Takuya OSHIMA, Ph.D. > Faculty of Engineering, Niigata University > 8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN > > From: Guenter Bartsch <[email protected]> > Subject: Re: [Paraview] Error when reading large XMLUnstructuredData file > Date: Tue, 4 Jan 2011 12:10:27 +0100 > >> Takuya, >> >> thanks for your quick reply. however, we do have set >> >> VTK_USE_64BIT_IDS ON >> >> and this is a 64bit linux platform (CentOS/RHEL x86_64) in case that >> matters - so unfortunately that isn't the solution here. >> >> thanks anyway and kind regards, >> >> guenter >> >> On Tue, Jan 4, 2011 at 12:06 PM, Takuya OSHIMA >> <[email protected]> wrote: >>> Hi Guenter and all, >>> >>> I also had the same issue with .vti (XML vtkImageData) files of >>> roughly the same sizes of around 3 GBytes. Currently I'm working >>> around the issue by building VTK and ParaView with VTK_USE_64BIT_IDS >>> set to ON, which will allow you to load up to 4 GBytes as compared to >>> 2 GBytes when VTK_USE_64BIT_IDS is OFF. >>> >>> However I'd be interested if there's more fundamental way of resolving >>> the issue because I'll be handling larger cases soon. Is there a >>> format (other than series of legacy VTK format files) that is >>> time-aware, parallelizable and allows more than 4 GBytes of >>> vtkImageData per partition? >>> >>> Takuya >>> >>> Takuya OSHIMA, Ph.D. >>> Faculty of Engineering, Niigata University >>> 8050 Ikarashi-Ninocho, Nishi-ku, Niigata, 950-2181, JAPAN >>> >>> From: Guenter Bartsch <[email protected]> >>> Subject: [Paraview] Error when reading large XMLUnstructuredData file >>> Date: Tue, 4 Jan 2011 11:45:18 +0100 >>> >>>> hi all, >>>> >>>> we have run into a problem reading a pretty large (3.6 GB) .vtu file. >>>> the file was produced by a fortran program that writes vtk data in the >>>> old ascii format which results in a 17G ascii file which takes very >>>> long to load in paraview (and results in an error message). so, we >>>> have written a converter program that reads the ascii file and outputs >>>> xml: >>>> >>>> >>>> int main(int argc, char *argp[]) >>>> { >>>> if (argc != 2) { >>>> cerr << "usage: " << argp[0] << " <vtk filename>\n\n"; >>>> return 1; >>>> } >>>> >>>> char *filename = argp[1]; >>>> >>>> cout << "Input filename: " << filename << "\n"; >>>> >>>> vtkSmartPointer<vtkUnstructuredGridReader> reader = >>>> vtkSmartPointer<vtkUnstructuredGridReader>::New(); >>>> reader->SetFileName(filename); >>>> >>>> char *ofilename = "foo.vtu"; >>>> cout << "Output filename: " << ofilename << "\n"; >>>> vtkSmartPointer<vtkXMLUnstructuredGridWriter> writer = >>>> vtkSmartPointer<vtkXMLUnstructuredGridWriter>::New(); >>>> writer->SetFileName(ofilename); >>>> writer->SetInputConnection(reader->GetOutputPort()); >>>> writer->Write(); >>>> >>>> return EXIT_SUCCESS; >>>> } >>>> >>>> this program runs fine. now, if i open the resulting file in paraview, >>>> i run into this error as soon as i hit apply: >>>> >>>> ERROR: In >>>> /opt/paraview-3.8.1/src/ParaView-3.8.1/VTK/IO/vtkXMLUnstructuredDataReader.cxx, >>>> line 652 >>>> vtkXMLUnstructuredGridReader (0x143f0c40): Cannot read cell >>>> connectivity from Cells in piece 0 because the "connectivity" array is >>>> not long enough. >>>> >>>> the converter did work on a smaller test data set and paraview could >>>> open it - only now with the large data set paraview fails. here is the >>>> header of the failing file: >>>> >>>> <?xml version="1.0"?> >>>> <VTKFile type="UnstructuredGrid" version="0.1" >>>> byte_order="LittleEndian" compressor="vtkZLibDataCompressor"> >>>> <UnstructuredGrid> >>>> <Piece NumberOfPoints="106496000" >>>> NumberOfCells="105122745" > >>>> <PointData Vectors="vectors"> >>>> <DataArray type="Float32" Name="vectors" >>>> NumberOfComponents="3" format="appended" RangeMin="0.0021820231521" >>>> RangeMax="24.459394872" offset="0" /> >>>> </PointData> >>>> <CellData> >>>> </CellData> >>>> <Points> >>>> <DataArray type="Float32" Name="Points" NumberOfComponents="3" >>>> format="appended" RangeMin="0" >>>> RangeMax="5007.1818403" offset="1580185620" /> >>>> </Points> >>>> <Cells> >>>> <DataArray type="Int64" Name="connectivity" format="appended" >>>> RangeMin="" RangeMax="" >>>> offset="2045521660" /> >>>> <DataArray type="Int64" Name="offsets" format="appended" >>>> RangeMin="" RangeMax="" >>>> offset="3504937336" /> >>>> <DataArray type="UInt8" Name="types" format="appended" >>>> RangeMin="" RangeMax="" >>>> offset="3671971228" /> >>>> </Cells> >>>> </Piece> >>>> </UnstructuredGrid> >>>> <AppendedData encoding="base64"> >>>> _WJgAAACAAAAAAAAAvXYAALF2AADJdgAAUHcAA... >>>> >>>> does anyone know what could be going on here? do we need to configure >>>> paraview to handle larger files? is this a bug? could the original >>>> ascii data set have been corrupt and the error reflects now in the xml >>>> file? >>>> >>>> thanks in advance and best regards, >>>> >>>> guenter >>>> _______________________________________________ >>>> Powered by www.kitware.com >>>> >>>> Visit other Kitware open-source projects at >>>> http://www.kitware.com/opensource/opensource.html >>>> >>>> Please keep messages on-topic and check the ParaView Wiki at: >>>> http://paraview.org/Wiki/ParaView >>>> >>>> Follow this link to subscribe/unsubscribe: >>>> http://www.paraview.org/mailman/listinfo/paraview >>> > _______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
