On Thu, Sep 14, 2017 at 10:35 AM, Barry Smith <bsm...@mcs.anl.gov> wrote:
> > > On Sep 14, 2017, at 11:10 AM, Kong, Fande <fande.k...@inl.gov> wrote: > > > > > > > > On Thu, Sep 14, 2017 at 9:47 AM, Matthew Knepley <knep...@gmail.com> > wrote: > > On Thu, Sep 14, 2017 at 11:43 AM, Adriano Côrtes <adrimacor...@gmail.com> > wrote: > > Dear Matthew, > > > > Thank you for your return. It worked, but this prompts another question. > So why PetscViewer does not write both files (.h5 and .xmf) directly, > instead of having to post-proc the .h5 file (in serial)? > > > > 1) Maintenance: Changing the Python is much easier than changing the C > you would add to generate it > > > > 2) Performance: On big parallel system, writing files is expensive so I > wanted to minimize what I had to do. > > > > 3) Robustness: Moving 1 file around is much easier than remembering 2. I > just always regenerate the xdmf when needed. > > > > And what about big 3D simulations? PETSc always serialize the output of > the distributed dmplex? Is there a way to output one .h5 per mesh partition? > > > > Given the way I/O is structured on big machines, we believe the multiple > file route is a huge mistake. Also, all our measurements > > say that sending some data on the network is not noticeable given the > disk access costs. > > > > I have slightly different things here. We tried the serial output, it > looks really slow for large-scale problems, and the first processor often > runs out of memory because of gathering all data from other processor cores. > > Where in PETSc is this? What type of viewer? Is there an example that > reproduces the problem? Even when we do not use MPI IO in PETSc we attempt > to not "put the entire object on the first process" so memory should not be > an issue. For example VecVew() should memory scale both with or without MPI > IO > We manually gather all data to the first processor core, and write it as a single vtk file. > > > > The parallel IO runs smoothly and much faster than I excepted. We have > done experiments with ten thousands of cores for a problem with 1 billion > of unknowns. > > Is this your own canned IO or something in PETSc? > We implement the writer based on the ISView and VecView with HDF5 viewer in PETSc to output all data as a single HDF. ISView and VecView do the magic job for me. > > > I did not see any concern so far. > > Ten thousand files is possibly manageable but I question 2 million. > Just one single HDF5 file. Fande, > > > > > > > Fande, > > > > > > Thanks, > > > > Matt > > > > Best regards, > > Adriano. > > > > > > 2017-09-14 12:00 GMT-03:00 Matthew Knepley <knep...@gmail.com>: > > On Thu, Sep 14, 2017 at 10:30 AM, Adriano Côrtes <adrimacor...@gmail.com> > wrote: > > Dear all, > > > > I am running the SNES ex12 and I'm passing the options -dm_view > hdf5:sol.h5 -vec_view hdf5:sol.h5::append to generate an output file. The > .h5 file is generated, but I'm not being able to load it in Paraview > (5.4.0-64bits). Paraview recognizes the file and offers severel options to > read it, here is the complete list > > > > Chombo Files > > GTC Files > > M3DC1 Files > > Multilevel 3D Plasma Files > > PFLOTRAN Files > > Pixie Files > > Tetrad Files > > UNIC Files > > VizSchema Files > > > > The problem is none of the options above work :( > > I'm using the configure option '-download-hdf5' and it installs hdf5 > version 1.8.18 > > Any hint of how to fix it and have the visualization working? > > > > Yes, Paraview does not directly read HDF5. It needs you to tell it what > the data in the HDF5 file means. You do > > this by creating a *.xdmf file, which is XML. We provide a tool > > > > $PETSC_DIR/bin/petsc_gen_xdmf.py <HDF5 file> > > > > which should automatically produce this file for you. Let us know if it > does not work. > > > > Thanks, > > > > Matt > > > > > > Best regards, > > Adriano. > > > > -- > > Adriano Côrtes > > ================================================= > > Campus Duque de Caxias and > > High-performance Computing Center (NACAD/COPPE) > > Federal University of Rio de Janeiro (UFRJ) > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www. > caam.rice.edu_-7Emk51_&d=DwIFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_ > _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m= > YTLjMkjfS0tYLZ3RxmJFoe8BT56h48axFCzaadZwBXA&s= > iLsaHQugaY4gj4DKE9gq8XdBt7q3ejdpDRfJ8RFerE0&e= > > > > > > > > -- > > Adriano Côrtes > > ================================================= > > Campus Duque de Caxias and > > High-performance Computing Center (NACAD/COPPE) > > Federal University of Rio de Janeiro (UFRJ) > > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > > -- Norbert Wiener > > > > https://urldefense.proofpoint.com/v2/url?u=http-3A__www. > caam.rice.edu_-7Emk51_&d=DwIFaQ&c=54IZrppPQZKX9mLzcGdPfFD1hxrcB_ > _aEkJFOKJFd00&r=DUUt3SRGI0_JgtNaS3udV68GRkgV4ts7XKfj2opmiCY&m= > YTLjMkjfS0tYLZ3RxmJFoe8BT56h48axFCzaadZwBXA&s= > iLsaHQugaY4gj4DKE9gq8XdBt7q3ejdpDRfJ8RFerE0&e= > > > >