>> Have you tried using the MPI-POSIX VFD for independent access? >Thanks - I'll report back with further findings
Rubbish! I still only get 150MB/s with the mpiposix driver. As Queen Victoria would have said "we are not amused" I suspect I've got an error somewhere because something should have changed. JB -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Mark Howison Sent: 05 April 2011 17:19 To: HDF Users Discussion List Subject: Re: [Hdf-forum] Chunking : was Howison et al Lustre mdc_config Hi John, What platform and parallel file system is this on? Have you tried using the MPI-POSIX VFD for independent access? Thanks, Mark On Tue, Apr 5, 2011 at 11:16 AM, Biddiscombe, John A. <[email protected]> wrote: > Elena, > > I was just replying to myself when your email came in. > I knocked up a quick wrapper to enable testing of the cache eviction stuff so > I'm happy (ish). > > However : I'm seeing puzzling behaviour with chunking. > Using collective IO, (tweaking various params) I see transfer rates from > 1GB/s to 3GB/s depending on stripe size, number of cb_nodes, etc. > > However, when using chunking with independent IO, I set stripe size to 6MB, > chunk dims to match the 6MB, each node is writing 6MB, alignment is set to > 6MB intervals and I've followed all the tips I can find. I see (for 512 nodes > writing 6MB each = 4GB total) a max throughput of around 150MB/s. > > This is shockingly slow compared to collective IO and I'm quite surprised. > I've been playing with this for a few days now and my general impression is > that > chunking = rubbish > collective = nice > > I did not expect chunking to be so bad compared to collective (which is a > shame as I was hoping to use it for compression etc). > > Can anyone suggest further tweaks that I should be looking out for to > change. (one thing for example that seems to make no difference is the > H5Pset_istore_k(fcpl, btree_ik); stuff. I still don't quite know what the > correct value for btree_ik is. Ive read the man page, but I'm puzzled as to > the correct meaning. if I know there will be 512 chunks, what is the 'right' > value of btree_ik? > > Any clues gratefully received for optimizing chunking. I hoped the thread > about 30,000 processes would carry on as I found it interesting to follow. > > ttfn > > JB > > > > -----Original Message----- > From: [email protected] [mailto:[email protected]] > On Behalf Of Elena Pourmal > Sent: 05 April 2011 17:01 > To: HDF Users Discussion List > Subject: Re: [Hdf-forum] Question re: Howison et al Lustre mdc_config tuning > recommendations > > Hi John, > On Apr 5, 2011, at 4:21 AM, Biddiscombe, John A. wrote: > >>>>> H5AC_cache_config_t mdc_config; >>>>> hid_t file_id; >>>>> file_id = H5Fopen("file.h5", H5ACC_RDWR, H5P_DEFAULT); >>>>> mdc_config.version = H5AC__CURR_CACHE_CONFIG_VERSION; >>>>> H5Pget_mdc_config(file_id, &mdc_config) >>>>> mdc_config.evictions_enabled = 0 /* FALSE */; >>>>> mdc_config.incr_mode = H5C_incr__off; >>>>> mdc_config.decr_mode = H5C_decr__off; >>>>> H5Pset_mdc_config(file_id, &mdc_config); >> >> I couldn't find fortran bindings for these. Do they exist in any recent >> releases or svn branches. >> > Fortran wrappers for do not exist. Please let us know which Fortran wrappers > do you need and we will add them to our to-do list. > > Elena >> thanks >> >> JB >> >> _______________________________________________ >> Hdf-forum is for HDF software users discussion. >> [email protected] >> http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > > _______________________________________________ > Hdf-forum is for HDF software users discussion. > [email protected] > http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org > _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org _______________________________________________ Hdf-forum is for HDF software users discussion. [email protected] http://mail.hdfgroup.org/mailman/listinfo/hdf-forum_hdfgroup.org
