Re: [OMPI users] Romio and OpenMPI builds
On Fri, Dec 07, 2012 at 11:44:05PM -0500, Eric Chamberland wrote: > >--with-io-romio-flags='--with-file-system=testfs+ufs+nfs+lustre' > exactly. > > > >We have Lustre, local filesystems (ufs), and NFSv3 and NFSv4 clients. So > >that list should be good for our site. > > If you have any choice at all please do not enable NFS. It will work, barely, but I'd rather see your users treat NFS as the broken horrorshow it is (with respect to MPI-IO) than to use it at all. ==rob > >Would this be a good recommendation for us to include in all our MPI builds? > I think yes, it is in the right direction, but I am not an > "expert"... some expert advice should be welcome. > > Eric > > > > >Brock Palen > >www.umich.edu/~brockp > >CAEN Advanced Computing > >bro...@umich.edu > >(734)936-1985 > > > > > > > >On Dec 3, 2012, at 7:12 PM, Eric Chamberland wrote: > > > >>Le 12/03/2012 05:37 PM, Brock Palen a écrit : > >>>I was trying to use hints with ROMIO and lustre prompted by another post > >>>on this list. > >>> > >>>I have a simple MPI-IO code and I cannot using the notes I find set the > >>>lustre striping using the config file and setting ROMIO_HINTS. > >>> > >>>Question: > >>> > >>>How can I check which ADIO drivers ROMIO in OPenMPI was built with when I > >>>built it? > >>>Can I make ROMIO go into 'verbose' mode and have it print what it is > >>>setting all its values to? > >>Try "ompi_info -a" and check for lustre in the output: > >> > >>ompi_info -a | grep -i romio > >>... > >> MCA io: information "io_romio_user_configure_params" > >> (value:<--with-file-system=testfs+ufs+nfs+lustre>, data source: default > >> value) > >> User-specified command line parameters passed to > >> ROMIO's configure script > >> MCA io: information "io_romio_complete_configure_params" > >> (value:<--with-file-system=testfs+ufs+nfs+lustre CFLAGS='-DNDEBUG -O3 > >> -xHOST -Wall -finline-functions -fno-strict-aliasing -restrict -pthread' > >> CPPFLAGS=' > >> -I/clumeq/src/Open-MPI/1.6.3/intel/openmpi-1.6.3/opal/mca/hwloc/hwloc132/hwloc/include > >> > >> -I/clumeq/src/Open-MPI/1.6.3/intel/Build/opal/mca/hwloc/hwloc132/hwloc/include > >>-I/usr/include/infiniband -I/usr/include/infiniband' FFLAGS='' > >> LDFLAGS=' ' --enable-shared --enable-static > >> --with-file-system=testfs+ufs+nfs+lustre > >> --prefix=/software/MPI/openmpi/1.6.3_intel --with-mpi=open_mpi > >> --disable-aio>, data source: default value) > >> Complete set of command line parameters passed to > >> ROMIO's configure script > >> > >>Eric > >> > >>___ > >>users mailing list > >>us...@open-mpi.org > >>http://www.open-mpi.org/mailman/listinfo.cgi/users > > > >___ > >users mailing list > >us...@open-mpi.org > >http://www.open-mpi.org/mailman/listinfo.cgi/users > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA
Re: [OMPI users] Romio and OpenMPI builds
Thanks! So it looks like most OpenMPI builds out there are running with ROMIO's that are obvious to any optimizations to what they are running on. I have added this to our build notes so we get it in next time. Thanks! Brock Palen www.umich.edu/~brockp CAEN Advanced Computing bro...@umich.edu (734)936-1985 On Dec 7, 2012, at 11:44 PM, Eric Chamberland wrote: > Hi Brock, > > Le 12/06/2012 05:10 PM, Brock Palen a écrit : >> Eric, >> >> You are correct, our builds do not show lustre support: >> >> MCA io: information "io_romio_user_configure_params" (value:, data >> source: default value) > surprise! ;-) >> >> So to enable this, when I build OpenMPI I should pass: >> >> --with-io-romio-flags='--with-file-system=testfs+ufs+nfs+lustre' > exactly. >> >> We have Lustre, local filesystems (ufs), and NFSv3 and NFSv4 clients. So >> that list should be good for our site. >> >> Would this be a good recommendation for us to include in all our MPI builds? > I think yes, it is in the right direction, but I am not an "expert"... some > expert advice should be welcome. > > Eric > >> >> Brock Palen >> www.umich.edu/~brockp >> CAEN Advanced Computing >> bro...@umich.edu >> (734)936-1985 >> >> >> >> On Dec 3, 2012, at 7:12 PM, Eric Chamberland wrote: >> >>> Le 12/03/2012 05:37 PM, Brock Palen a écrit : I was trying to use hints with ROMIO and lustre prompted by another post on this list. I have a simple MPI-IO code and I cannot using the notes I find set the lustre striping using the config file and setting ROMIO_HINTS. Question: How can I check which ADIO drivers ROMIO in OPenMPI was built with when I built it? Can I make ROMIO go into 'verbose' mode and have it print what it is setting all its values to? >>> Try "ompi_info -a" and check for lustre in the output: >>> >>> ompi_info -a | grep -i romio >>> ... >>> MCA io: information "io_romio_user_configure_params" >>> (value:<--with-file-system=testfs+ufs+nfs+lustre>, data source: default >>> value) >>> User-specified command line parameters passed to >>> ROMIO's configure script >>> MCA io: information "io_romio_complete_configure_params" >>> (value:<--with-file-system=testfs+ufs+nfs+lustre CFLAGS='-DNDEBUG -O3 >>> -xHOST -Wall -finline-functions -fno-strict-aliasing -restrict -pthread' >>> CPPFLAGS=' >>> -I/clumeq/src/Open-MPI/1.6.3/intel/openmpi-1.6.3/opal/mca/hwloc/hwloc132/hwloc/include >>> >>> -I/clumeq/src/Open-MPI/1.6.3/intel/Build/opal/mca/hwloc/hwloc132/hwloc/include >>>-I/usr/include/infiniband -I/usr/include/infiniband' FFLAGS='' LDFLAGS=' >>> ' --enable-shared --enable-static --with-file-system=testfs+ufs+nfs+lustre >>> --prefix=/software/MPI/openmpi/1.6.3_intel --with-mpi=open_mpi >>> --disable-aio>, data source: default value) >>> Complete set of command line parameters passed to >>> ROMIO's configure script >>> >>> Eric >>> >>> ___ >>> users mailing list >>> us...@open-mpi.org >>> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> ___ >> users mailing list >> us...@open-mpi.org >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] Romio and OpenMPI builds
Hi Brock, Le 12/06/2012 05:10 PM, Brock Palen a écrit : Eric, You are correct, our builds do not show lustre support: MCA io: information "io_romio_user_configure_params" (value:, data source: default value) surprise! ;-) So to enable this, when I build OpenMPI I should pass: --with-io-romio-flags='--with-file-system=testfs+ufs+nfs+lustre' exactly. We have Lustre, local filesystems (ufs), and NFSv3 and NFSv4 clients. So that list should be good for our site. Would this be a good recommendation for us to include in all our MPI builds? I think yes, it is in the right direction, but I am not an "expert"... some expert advice should be welcome. Eric Brock Palen www.umich.edu/~brockp CAEN Advanced Computing bro...@umich.edu (734)936-1985 On Dec 3, 2012, at 7:12 PM, Eric Chamberland wrote: Le 12/03/2012 05:37 PM, Brock Palen a écrit : I was trying to use hints with ROMIO and lustre prompted by another post on this list. I have a simple MPI-IO code and I cannot using the notes I find set the lustre striping using the config file and setting ROMIO_HINTS. Question: How can I check which ADIO drivers ROMIO in OPenMPI was built with when I built it? Can I make ROMIO go into 'verbose' mode and have it print what it is setting all its values to? Try "ompi_info -a" and check for lustre in the output: ompi_info -a | grep -i romio ... MCA io: information "io_romio_user_configure_params" (value:<--with-file-system=testfs+ufs+nfs+lustre>, data source: default value) User-specified command line parameters passed to ROMIO's configure script MCA io: information "io_romio_complete_configure_params" (value:<--with-file-system=testfs+ufs+nfs+lustre CFLAGS='-DNDEBUG -O3 -xHOST -Wall -finline-functions -fno-strict-aliasing -restrict -pthread' CPPFLAGS=' -I/clumeq/src/Open-MPI/1.6.3/intel/openmpi-1.6.3/opal/mca/hwloc/hwloc132/hwloc/include -I/clumeq/src/Open-MPI/1.6.3/intel/Build/opal/mca/hwloc/hwloc132/hwloc/include -I/usr/include/infiniband -I/usr/include/infiniband' FFLAGS='' LDFLAGS=' ' --enable-shared --enable-static --with-file-system=testfs+ufs+nfs+lustre --prefix=/software/MPI/openmpi/1.6.3_intel --with-mpi=open_mpi --disable-aio>, data source: default value) Complete set of command line parameters passed to ROMIO's configure script Eric ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users ___ users mailing list us...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] Romio and OpenMPI builds
Eric, You are correct, our builds do not show lustre support: MCA io: information "io_romio_user_configure_params" (value: , data source: default value) So to enable this, when I build OpenMPI I should pass: --with-io-romio-flags='--with-file-system=testfs+ufs+nfs+lustre' We have Lustre, local filesystems (ufs), and NFSv3 and NFSv4 clients. So that list should be good for our site. Would this be a good recommendation for us to include in all our MPI builds? Brock Palen www.umich.edu/~brockp CAEN Advanced Computing bro...@umich.edu (734)936-1985 On Dec 3, 2012, at 7:12 PM, Eric Chamberland wrote: > Le 12/03/2012 05:37 PM, Brock Palen a écrit : >> I was trying to use hints with ROMIO and lustre prompted by another post on >> this list. >> >> I have a simple MPI-IO code and I cannot using the notes I find set the >> lustre striping using the config file and setting ROMIO_HINTS. >> >> Question: >> >> How can I check which ADIO drivers ROMIO in OPenMPI was built with when I >> built it? >> Can I make ROMIO go into 'verbose' mode and have it print what it is setting >> all its values to? > Try "ompi_info -a" and check for lustre in the output: > > ompi_info -a | grep -i romio > ... > MCA io: information "io_romio_user_configure_params" (value: > <--with-file-system=testfs+ufs+nfs+lustre>, data source: default value) > User-specified command line parameters passed to > ROMIO's configure script > MCA io: information "io_romio_complete_configure_params" > (value: <--with-file-system=testfs+ufs+nfs+lustre CFLAGS='-DNDEBUG -O3 > -xHOST -Wall -finline-functions -fno-strict-aliasing -restrict -pthread' > CPPFLAGS=' > -I/clumeq/src/Open-MPI/1.6.3/intel/openmpi-1.6.3/opal/mca/hwloc/hwloc132/hwloc/include > > -I/clumeq/src/Open-MPI/1.6.3/intel/Build/opal/mca/hwloc/hwloc132/hwloc/include >-I/usr/include/infiniband -I/usr/include/infiniband' FFLAGS='' LDFLAGS=' ' > --enable-shared --enable-static --with-file-system=testfs+ufs+nfs+lustre > --prefix=/software/MPI/openmpi/1.6.3_intel --with-mpi=open_mpi > --disable-aio>, data source: default value) > Complete set of command line parameters passed to > ROMIO's configure script > > Eric > > ___ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users
Re: [OMPI users] Romio and OpenMPI builds
Le 12/03/2012 05:37 PM, Brock Palen a écrit : I was trying to use hints with ROMIO and lustre prompted by another post on this list. I have a simple MPI-IO code and I cannot using the notes I find set the lustre striping using the config file and setting ROMIO_HINTS. Question: How can I check which ADIO drivers ROMIO in OPenMPI was built with when I built it? Can I make ROMIO go into 'verbose' mode and have it print what it is setting all its values to? Try "ompi_info -a" and check for lustre in the output: ompi_info -a | grep -i romio ... MCA io: information "io_romio_user_configure_params" (value: <--with-file-system=testfs+ufs+nfs+lustre>, data source: default value) User-specified command line parameters passed to ROMIO's configure script MCA io: information "io_romio_complete_configure_params" (value: <--with-file-system=testfs+ufs+nfs+lustre CFLAGS='-DNDEBUG -O3 -xHOST -Wall -finline-functions -fno-strict-aliasing -restrict -pthread' CPPFLAGS=' -I/clumeq/src/Open-MPI/1.6.3/intel/openmpi-1.6.3/opal/mca/hwloc/hwloc132/hwloc/include -I/clumeq/src/Open-MPI/1.6.3/intel/Build/opal/mca/hwloc/hwloc132/hwloc/include -I/usr/include/infiniband -I/usr/include/infiniband' FFLAGS='' LDFLAGS=' ' --enable-shared --enable-static --with-file-system=testfs+ufs+nfs+lustre --prefix=/software/MPI/openmpi/1.6.3_intel --with-mpi=open_mpi --disable-aio>, data source: default value) Complete set of command line parameters passed to ROMIO's configure script Eric
[OMPI users] Romio and OpenMPI builds
I was trying to use hints with ROMIO and lustre prompted by another post on this list. I have a simple MPI-IO code and I cannot using the notes I find set the lustre striping using the config file and setting ROMIO_HINTS. Question: How can I check which ADIO drivers ROMIO in OPenMPI was built with when I built it? Can I make ROMIO go into 'verbose' mode and have it print what it is setting all its values to? Thanks! Brock Palen www.umich.edu/~brockp CAEN Advanced Computing bro...@umich.edu (734)936-1985
