Dear HPX developers and users,
I work on a modelling framework which is used by domain experts like
hydrologists and soil scientists to build forward simulation models. HPX
is used in the implementation to distribute computations over CPU cores
and compute nodes.
Typically, large amounts of data are being read and written, not only at
the start and end of a simulation, but also while iterating through
time. We mostly work with gridded data (rasters).
I/O has become a performance and scalability bottleneck and I am looking
at ways to solve this. Together with a parallel I/O expert I have
started working on combining AMT / HPX with parallel I/O. Technically
this works now, using HDF5 as a data format, and on a GPFS file system.
Our current approach is quite simple. Instead of performing computations
on partitions of data in HPX threads, we perform (independent) I/O. We
plan on performing performance and scalability experiments in the near
future.
To prevent that we miss something important I would like to ask if you
have information that is related specifically to *combining AMT / HPX
with parallel I/O*, for example:
- HPX facilities that are useful in this context
- Lessons learned doing something similar
- Pointers to code that does something similar
Thanks in advance for any insights!
Kind regards,
Kor
_______________________________________________
hpx-users mailing list
[email protected]
https://mail.cct.lsu.edu/mailman/listinfo/hpx-users
