No problem. It wasn't much of a delay. The scenario involves a combination of MPI and OpenMP (or other threading scheme). Basically, the software will launch one or more processes via MPI, which then spawn threads to do the work.
What we've been seeing is that, without something like '--bind-to none' or similar, those threads end up being pinned to the same processor as the process that spawned them. We're okay with a bind=none, since we already have cgroups in place to constrain the user to the resources they request. We might get more process/thread migration between processors (but within the cgroup) than we would like, but that's still probably acceptable in this scenario. If there's a better solution, we'd love to hear it. Lloyd Brown Systems Administrator Fulton Supercomputing Lab Brigham Young University http://marylou.byu.edu On 11/03/2015 08:16 AM, Ralph Castain wrote: > Sorry for delay - was on travel. > > hwloc_base_binding_policy=none > > Alternatively, you may get better performance if you bind to numa or > socket levels, assuming you want one proc per socket: > > hwloc_base_binding_policy=socket [or numa] > rmaps_base_mapping_policy=socket [or numa] > > HTH > Ralph > >> On Nov 2, 2015, at 8:31 AM, Lloyd Brown <lloyd_br...@byu.edu >> <mailto:lloyd_br...@byu.edu>> wrote: >> >> Is there an environment variable option, as well as the >> openmpi-mca-params.conf to set the equivalent of "--bind-to none"? >> Similar to how I can specify the environment variable >> "OMPI_MCA_btl=^openib" instead of the cli param "--mca btl ^openib"? >> >> We're running into a situation where users have a combination of OpenMPI >> and OpenMP threads, and the threads get constrained to the same >> processor where the OpenMPI process was launched. As far as we can >> tell, this started with v1.8.x. >> >> >> Lloyd Brown >> Systems Administrator >> Fulton Supercomputing Lab >> Brigham Young University >> http://marylou.byu.edu <http://marylou.byu.edu/> >> >> On 10/01/2015 09:02 AM, Nick Papior wrote: >>> You can define default mca parameters in this file: >>> <prefix>/etc/openmpi-mca-params.conf >>> >>> 2015-10-01 16:57 GMT+02:00 Grigory Shamov >>> <grigory.sha...@umanitoba.ca <mailto:grigory.sha...@umanitoba.ca> >>> <mailto:grigory.sha...@umanitoba.ca>>: >>> >>> Hi All, >>> >>> A parhaps naive question: is it possible to set ' mpiexec —bind-to >>> none ' as a system-wide default in 1.10, like, by setting an >>> OMPI_xxx variable? >>> >>> -- >>> Grigory Shamov >>> Westgrid/ComputeCanada Site Lead >>> University of Manitoba >>> E2-588 EITC Building, >>> (204) 474-9625 <tel:%28204%29%20474-9625> >>> >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org >>> <mailto:us...@open-mpi.org> <mailto:us...@open-mpi.org> >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2015/10/27764.php >>> >>> >>> >>> >>> -- >>> Kind regards Nick >>> >>> >>> _______________________________________________ >>> users mailing list >>> us...@open-mpi.org <mailto:us...@open-mpi.org> >>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >>> Link to this post: >>> http://www.open-mpi.org/community/lists/users/2015/10/27765.php >>> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org <mailto:us...@open-mpi.org> >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this >> post: http://www.open-mpi.org/community/lists/users/2015/11/27974.php > > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/11/27978.php >
