Mohamad,
I really do not understand how to reply to this forum :(. I tried to
reply to your post, which I received via e-mail. In this e-mail, there
was the following note:
"
If you reply to this email, your message will be added to the discussion
below:
http://hdf-forum.184993.n3.nabble.com/Very-poor-performance-of-pHDF5-when-using-single-shared-file-tp4026443p4026449.html
"
So, I replied to this e-mail, and received another one:
Subject: Delivery Status Notification (Failure)
"
Delivery to the following recipient failed permanently:
[email protected]
Your email to [email protected] has been rejected
because you are not allowed to post to
http://hdf-forum.184993.n3.nabble.com/Very-poor-performance-of-pHDF5-when-using-single-shared-file-tp4026443p4026449.html
. Please contact the owner about permissions or visit the Nabble Support
forum.
"
What the hell... why does it say I should reply and then that I am not
allowed to post to my own thread???
Anyway, I tried to post the following information:
I did some experiments yesterday using the BlueWaters cluster. The
stripe count is limited there to 160. For runs with 256 MPI
processes/cores and fixed datasets were the writing times:
separate files: 1.36 [s]
single file, 1 stripe: 133.6 [s]
single file, best result: 17.2 [s]
(I did multiple runs with various combinations of strip count and size,
presenting the best results I have obtained.)
Increasing the number of stripes obviously helped a lot, but comparing
with the separate-files strategy, the writing time is still more than
ten times slower . Do you think it is "normal"?
Might chunking help here?
Thanks,
Daniel
Dne 30. 8. 2013 16:05, Daniel Langr napsal(a):
I've run some benchmark, where within an MPI program, each process wrote
3 plain 1D arrays to 3 datasets of an HDF5 file. I've used the following
writing strategies:
1) each process writes to its own file,
2) each process writes to the same file to its own dataset,
3) each process writes to the same file to a same dataset.
I've tested 1)-3) for both fixed/chunked datasets (chunk size 1024), and
I've tested 2)-3) for both independent/collective options of the MPI
driver. I've also used 3 different clusters for measurements (all quite
modern).
As a result, the running (storage) times of the same-file strategy, i.e.
2) and 3), were of orders of magnitudes longer than the running times of
the separate-files strategy. For illustration:
cluster #1, 512 MPI processes, each process stores 100 MB of data, fixed
data sets:
1) separate files: 2.73 [s]
2) single file, independent calls, separate data sets: 88.54[s]
cluster #2, 256 MPI processes, each process stores 100 MB of data,
chunked data sets (chunk size 1024):
1) separate files: 10.40 [s]
2) single file, independent calls, shared data sets: 295 [s]
3) single file, collective calls, shared data sets: 3275 [s]
Any idea why the single-file strategy gives so poor writing performance?
Daniel
_______________________________________________
Hdf-forum is for HDF software users discussion.
[email protected]
http://mail.lists.hdfgroup.org/mailman/listinfo/hdf-forum_lists.hdfgroup.org
