Hello Lawrence,
Thanks for the swift response. I am sorry that mine has been much slower.
The information that you provided was useful.
I had been trying to generate the initial state using just one MPI
process. I rewrote my code slightly so that I could use more than one
process. Once I had done this I succeeded in generating an initial state
by employing two MPI processes.
I do not understand in detail why one process was insufficient because
it seems to me that I was within the stated limits (the maximum size of
file generated using two processes is 2.6GB and the number of elements
is less than 310,000,000) but I do not doubt that your suspicion is correct.
Best wishes,
David
On 26/11/2021 15:25, Lawrence Mitchell wrote:
This email was sent to you by someone outside the University.
You should only click on links or attachments if you are certain that
the email is genuine and the content is safe.
This is failing setting the chunksize:
https://gitlab.com/petsc/petsc/-/blob/main/src/dm/impls/da/gr2.c#L517
It is hard for me to follow this code, but it looks like the chunk is
just set to the total extent of the DA (on the process?). This can
grow too large for HDF5, which has limits described here
https://support.hdfgroup.org/HDF5/doc/Advanced/Chunking/
chunks must have fewer than 2^32-1 elements and a max size of 4GB
I suspect you’re hitting that limit.
I guess this part of the code should be fully refactored to set
sensible chunk sizes. That same document suggests 1MB is a good option.
Lawrence
On Fri, 26 Nov 2021 at 15:10, David Scott <[email protected]> wrote:
Hello,
I am trying to write out HDF5 files. The program I have works all
right
up to a point but then fails when I double the size of the file that I
am trying to write out. I have attached a file containing the error
message and another file containing the short program that I used to
generate it.
The first of these files reports that Petsc 3.14.2 was used. I
know that
this is old but I have obtained the same result with Petsc 3.16.1.
Also,
I have observed the same behaviour on two different machines. I tried
using 64 bit indices but it made no difference.
The program runs successfully with the following command line options
-global_dim_x 1344 -global_dim_y 336 -global_dim_z 336
but fails with these
-global_dim_x 2688 -global_dim_y 336 -global_dim_z 336
All the best,
David
The University of Edinburgh is a charitable body, registered in
Scotland, with registration number SC005336. Is e buidheann
carthannais a th’ ann an Oilthigh Dhùn Èideann, clàraichte an
Alba, àireamh clàraidh SC005336.
