Thanks, Dave. Mike, can you test that this branch works with your large problems? I tested that .vtu works in parallel for small problems, where works = loads correctly in Paraview and VisIt.
https://gitlab.com/petsc/petsc/-/merge_requests/6081 Dave May <dave.mayhe...@gmail.com> writes: > On Tue 14. Feb 2023 at 21:27, Jed Brown <j...@jedbrown.org> wrote: > >> Dave May <dave.mayhe...@gmail.com> writes: >> >> > On Tue 14. Feb 2023 at 17:17, Jed Brown <j...@jedbrown.org> wrote: >> > >> >> Can you share a reproducer? I think I recall the format requiring >> certain >> >> things to be Int32. >> > >> > >> > By default, the byte offset used with the appended data format is >> UInt32. I >> > believe that’s where the sizeof(int) is coming from. This default is >> > annoying as it limits the total size of your appended data to be < 3 GB. >> > That said, in the opening of the paraview file you can add this attribute >> > >> > header_type="UInt64" >> >> You mean in the header of the .vtu? > > > Yeah, within the open VTKFile tag. > Like this > < VTKFile type=“xxx”, byte_order="LittleEndian" header_type="UInt64" > > > Do you happen to have an example or pointers to docs describing this >> feature? > > > Example yes - will send it to you tomorrow. Docs… not really. Only stuff > like this > > https://kitware.github.io/paraview-docs/latest/python/paraview.simple.XMLPStructuredGridWriter.html > > > https://kitware.github.io/paraview-docs/v5.8.0/python/paraview.simple.XMLMultiBlockDataWriter.html > > All the writers seem to support it. > > > Can we always do this? > > > Yep! > > > It isn't mentioned in these: >> >> https://vtk.org/wp-content/uploads/2015/04/file-formats.pdf (PDF was >> created in 2003) >> >> https://kitware.github.io/vtk-examples/site/VTKFileFormats/#xml-file-formats >> > > Yes I know. I’ve tied myself in knots for years because the of the > assumption that the offset had to be an int. > > Credit for the discovery goes to Carsten Uphoff. He showed me this. > > Cheers, > Dave > > > >> > then the size of the offset is now UInt64 and now large files can be >> > finally written. >> > >> > >> > Cheers, >> > Dave >> > >> > >> > >> > >> >> >> >> Mike Michell <mi.mike1...@gmail.com> writes: >> >> >> >> > Thanks for the note. >> >> > I understood that PETSc calculates the offsets for me through >> "boffset" >> >> > variable in plexvtu.c file. Please correct me if it is wrong. >> >> > >> >> > If plexvtu.c has a bug, it could be around "write file header" part in >> >> > which the boffset is also computed. Is this correct? I am not using >> >> complex >> >> > number. >> >> > There are several mixed parts among "Int32, UInt8, PetscInt_FMT, >> >> > PetscInt64_FMT" in writing the header. >> >> > >> >> > Which combination of those flags is correct for 64bit indices? I am >> gonna >> >> > modify plexvtu.c file with "#if defined(PETSC_USE_64BIT_INDICES)" >> >> > statement, but I do not know what is the correct form of the header >> flag >> >> > for 64bit indices. >> >> > >> >> > It is also confusing to me: >> >> > boffset += gpiece[r].ncells * sizeof(PetscInt) + sizeof(int); >> >> > How is sizeof(PetscInt) different from sizeof(int)? >> >> > >> >> > Thanks, >> >> > Mike >> >> > >> >> > >> >> >> On Tue, Feb 14, 2023 at 11:45 AM Mike Michell <mi.mike1...@gmail.com >> > >> >> >> wrote: >> >> >> >> >> >>> I was trying to modify the header flags from "Int32" to "Int64", but >> >> the >> >> >>> problem was not resolved. Could I get any additional comments? >> >> >>> >> >> >> >> >> >> The calculated offsets are not correct I think. >> >> >> >> >> >> Matt >> >> >> >> >> >> >> >> >>> Thanks, >> >> >>> Mike >> >> >>> >> >> >>> >> >> >>>> Thanks for the comments. >> >> >>>> To be precise on the question, the entire part of the header of the >> >> .vtu >> >> >>>> file is attached: >> >> >>>> >> >> >>>> <?xml version="1.0"?> >> >> >>>> <VTKFile type="UnstructuredGrid" version="0.1" >> >> byte_order="LittleEndian"> >> >> >>>> <UnstructuredGrid> >> >> >>>> <Piece NumberOfPoints="4872" NumberOfCells="4000"> >> >> >>>> <Points> >> >> >>>> <DataArray type="Float64" Name="Position" >> >> NumberOfComponents="3" >> >> >>>> format="appended" offset="0" /> >> >> >>>> </Points> >> >> >>>> <Cells> >> >> >>>> <DataArray type="Int32" Name="connectivity" >> >> >>>> NumberOfComponents="1" format="appended" offset="116932" /> >> >> >>>> <DataArray type="Int32" Name="offsets" >> >> >>>> NumberOfComponents="1" format="appended" offset="372936" /> >> >> >>>> <DataArray type="UInt8" Name="types" >> >> >>>> NumberOfComponents="1" format="appended" offset="404940" /> >> >> >>>> </Cells> >> >> >>>> <CellData> >> >> >>>> <DataArray type="Int32" Name="Rank" NumberOfComponents="1" >> >> >>>> format="appended" offset="408944" /> >> >> >>>> </CellData> >> >> >>>> <PointData> >> >> >>>> <DataArray type="Float64" Name="Vec_0x37c89c0_4Field_0.0" >> >> >>>> NumberOfComponents="1" format="appended" offset="424948" /> >> >> >>>> </PointData> >> >> >>>> </Piece> >> >> >>>> <Piece NumberOfPoints="4872" NumberOfCells="4000"> >> >> >>>> <Points> >> >> >>>> <DataArray type="Float64" Name="Position" >> >> NumberOfComponents="3" >> >> >>>> format="appended" offset="463928" /> >> >> >>>> </Points> >> >> >>>> <Cells> >> >> >>>> <DataArray type="Int32" Name="connectivity" >> >> >>>> NumberOfComponents="1" format="appended" offset="580860" /> >> >> >>>> <DataArray type="Int32" Name="offsets" >> >> >>>> NumberOfComponents="1" format="appended" offset="836864" /> >> >> >>>> <DataArray type="UInt8" Name="types" >> >> >>>> NumberOfComponents="1" format="appended" offset="868868" /> >> >> >>>> </Cells> >> >> >>>> <CellData> >> >> >>>> <DataArray type="Int32" Name="Rank" NumberOfComponents="1" >> >> >>>> format="appended" offset="872872" /> >> >> >>>> </CellData> >> >> >>>> <PointData> >> >> >>>> <DataArray type="Float64" Name="Vec_0x37c89c0_4Field_0.0" >> >> >>>> NumberOfComponents="1" format="appended" offset="888876" /> >> >> >>>> </PointData> >> >> >>>> </Piece> >> >> >>>> </UnstructuredGrid> >> >> >>>> <AppendedData encoding="raw"> >> >> >>>> >> >> >>>> >> >> >>>> Thanks, >> >> >>>> Mike >> >> >>>> >> >> >>>> >> >> >>>>> On Sun, Feb 12, 2023 at 6:15 PM Mike Michell < >> mi.mike1...@gmail.com> >> >> >>>>> wrote: >> >> >>>>> >> >> >>>>>> Dear PETSc team, >> >> >>>>>> >> >> >>>>>> I am a user of PETSc with Fortran. My code uses DMPlex to handle >> dm >> >> >>>>>> object. To print out output variable and mesh connectivity, I use >> >> VecView() >> >> >>>>>> by defining PetscSection on that dm and borrow a vector. The type >> >> of the >> >> >>>>>> viewer is set to PETSCVIEWERVTK. >> >> >>>>>> >> >> >>>>>> With 32bit indices, the above work flow has no issue. However, if >> >> >>>>>> PETSc is configured with 64bit indices, my output .vtu file has >> an >> >> error if >> >> >>>>>> I open the file with visualization tools, such as Paraview or >> >> Tecplot, >> >> >>>>>> saying that: >> >> >>>>>> "Cannot read cell connectivity from Cells in piece 0 because the >> >> >>>>>> "offsets" array is not monotonically increasing or starts with a >> >> value >> >> >>>>>> other than 0." >> >> >>>>>> >> >> >>>>>> If I open the .vtu file from terminal, I can see such a line: >> >> >>>>>> ... >> >> >>>>>> <DataArray type="Int32" Name="connectivity" >> NumberOfComponents="1" >> >> >>>>>> format="appended" offset="580860" /> >> >> >>>>>> ... >> >> >>>>>> >> >> >>>>>> I expected "DataArray type="Int64", since the PETSc has 64bit >> >> indices. >> >> >>>>>> Could I get recommendations that I need to check to resolve the >> >> issue? >> >> >>>>>> >> >> >>>>> >> >> >>>>> This is probably a bug. We will look at it. >> >> >>>>> >> >> >>>>> Jed, I saw that Int32 is hardcoded in plexvtu.c, but >> sizeof(PetscInt) >> >> >>>>> is used to calculate the offset, which looks inconsistent. Can you >> >> take a >> >> >>>>> look? >> >> >>>>> >> >> >>>>> Thanks, >> >> >>>>> >> >> >>>>> Matt >> >> >>>>> >> >> >>>>> >> >> >>>>>> Thanks, >> >> >>>>>> Mike >> >> >>>>>> >> >> >>>>> >> >> >>>>> >> >> >>>>> -- >> >> >>>>> What most experimenters take for granted before they begin their >> >> >>>>> experiments is infinitely more interesting than any results to >> which >> >> their >> >> >>>>> experiments lead. >> >> >>>>> -- Norbert Wiener >> >> >>>>> >> >> >>>>> https://www.cse.buffalo.edu/~knepley/ >> >> >>>>> <http://www.cse.buffalo.edu/~knepley/> >> >> >>>>> >> >> >>>> >> >> >> >> >> >> -- >> >> >> What most experimenters take for granted before they begin their >> >> >> experiments is infinitely more interesting than any results to which >> >> their >> >> >> experiments lead. >> >> >> -- Norbert Wiener >> >> >> >> >> >> https://www.cse.buffalo.edu/~knepley/ >> >> >> <http://www.cse.buffalo.edu/~knepley/> >> >> >> >> >> >>
