Hi Mark,
Thank you for answering my question. You are right about the
practicality issue. This isn't a serious issue if the data is stored in a
distributed sense with some indicators specifying which processor has the
data. This data is treated as a background database and it doesn't move
during the transfer process.
The reason I asked this question is that I've seen HDF5 taking quite
sometime to read data in an unstructured manner while the MPI calls just
zip through. On an another note, are there any guidelines for parallel I/O
in case of unstructured data?
Regards
Suman
On Tue, Apr 9, 2013 at 10:39 AM, Miller, Mark C. <[email protected]> wrote:
> Hmm. If I understand the question, I really cannot imagine a scenario
> where parallel I/O would be "faster" than MPI send/recv calls.
>
> However, there may be a practicality issue here. It may be the case that
> the data processor k needs is on processor j but processor k doesn't know
> that processor j has it and processor j doesn't know that processor k needs
> it. So, there has to be some communication to for the processors to learn
> that.
>
> And, if those processors are 'somewhere else' in their execution, then
> you have a significant issue in programming to take advantage of the fact
> that you could use MPI send/recv to move the data anyways. In the end, it
> just might be more practical to just read data from the file, even if it is
> quite a bit slower.
>
> I am not sure I answered the question you asked though ;)
>
> Mark
>
> --
> Mark C. Miller, Lawrence Livermore National Laboratory
> ================!!LLNL BUSINESS ONLY!!================
> [email protected] urgent: [email protected]
> T:8-6 (925)-423-5901 M/W/Th:7-12,2-7 (530)-753-8511
>
> From: Suman Vajjala <[email protected]>
> Reply-To: HDF Users Discussion List <[email protected]>
> Date: Monday, April 8, 2013 9:38 PM
> To: HDF Users Discussion List <[email protected]>
> Subject: [Hdf-forum] Performance query
>
> Hi,
>
> I have a question regarding the performance of parallel I/O vs MPI
> communication based calls. I have data which needs to be accessed by
> different processors. If the data is in memory then MPI calls (Send/Recv)
> does the job. In an another scenario the data is written to a H5 file and
> different processors access the respective data using parallel I/O. Would
> MPI calls be faster than HDF5 parallel I/O? (data access could be
> unstructured)
>
> Regards
> Suman Vajjala
>
> _______________________________________________
> Hdf-forum is for HDF software users discussion.
> [email protected]
> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
>
>
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
