Re: [OMPI devel] accessors to context id and message id's
If I understand correctly your question, then we don't need any extension. Each request has a unique ID (from PERUSE perspective). However, if I remember well this is only half implemented in our PERUSE layer (i.e. it works only for expected requests). This should be quite easy to fix, if someone invest few hours into it. For the context id, a user can always use the c2f function to get the fortran ID (which for Open MPI is the communicator ID). Thanks, george. On Nov 5, 2007, at 8:01 AM, Terry Dontje wrote: Currently in order to do message tracing one either has to rely on some error prone postprocessing of data or replicating some MPI internals up in the PMPI layer. It would help Sun's tools group (and I believe U Dresden also) if Open MPI would create a couple APIs that exoposed the following: 1. PML Message ids used for a request 2. Context id for a specific communicator I could see a couple ways of providing this information. Either by extending the PERUSE probes or creating actual functions that one would pass in a request handle or communicator handle to get the appropriate data back. This is just a thought right now which why this email is not in an RFC format. I wanted to get a feel from the community as to the interest in such APIs and if anyone may have specific issues with us providing such interfaces. If the responses seems positive I will follow this message up with an RFC. thanks, --td ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel smime.p7s Description: S/MIME cryptographic signature
Re: [OMPI devel] Environment forwarding
Thanks for the clarification everyone.
Tim
On Monday 05 November 2007 05:41:00 pm Torsten Hoefler wrote:
> On Mon, Nov 05, 2007 at 05:32:04PM -0500, Brian W. Barrett wrote:
> > On Mon, 5 Nov 2007, Torsten Hoefler wrote:
> > > On Mon, Nov 05, 2007 at 04:57:19PM -0500, Brian W. Barrett wrote:
> > >> This is extremely tricky to do. How do you know which environment
> > >> variables to forward (foo in this case) and which not to (hostname).
> > >> SLURM has a better chance, since it's linux only and generally only
> > >> run on tightly controlled clusters. But there's a whole variety of
> > >> things that shouldn't be forwarded and that list differs from OS to
> > >> OS.
> > >>
> > >> I believe we toyed around with the "right thing" in LAM and early on
> > >> with OPen MPI and decided that it was too hard to meet expected
> > >> behavior.
> > >
> > > Some applications rely on this (I know at least two right away, Gamess
> > > and Abinit) and they work without problems with Lam/Mpich{1,2} but not
> > > with Open MPI. I am *not* arguing that those applications are correct
> > > (I agree that this way of passing arguments is ugly, but it's done).
> > >
> > > I know it's not defined in the standard but I think it's a nice
> > > convenient functionality. E.g., setting the LD_LIBRARY_PATH to find
> > > libmpi.so in the .bashrc is also a pain if you have multiple (Open)
> > > MPIs installed.
> >
> > LAM does not automatically propogate environment variables -- it's
> > behavior is almost *exactly* like Open MPI's. There might be a situation
> > where the environment is not quite so scrubbed if a process is started on
> > the same node mpirun is executed on, but it's only appearances -- in
> > reality, that's the environment that was alive when lamboot was executed.
>
> ok, I might have executed it on the same node (was a while ago).
>
> > With both LAM and Open MPI, there is the -x option to propogate a list of
> > environment variables, but that's about it. Neither will push
> > LD_LIBRARY_PATH by default (and there are many good reasons for that,
> > particularly in heterogeneous situations).
>
> Ah, heterogeneous! Yes, I agree.
>
> Torsten
Re: [OMPI devel] Environment forwarding
On Mon, Nov 05, 2007 at 05:32:04PM -0500, Brian W. Barrett wrote:
> On Mon, 5 Nov 2007, Torsten Hoefler wrote:
>
> > On Mon, Nov 05, 2007 at 04:57:19PM -0500, Brian W. Barrett wrote:
> >> This is extremely tricky to do. How do you know which environment
> >> variables to forward (foo in this case) and which not to (hostname).
> >> SLURM has a better chance, since it's linux only and generally only run on
> >> tightly controlled clusters. But there's a whole variety of things that
> >> shouldn't be forwarded and that list differs from OS to OS.
> >>
> >> I believe we toyed around with the "right thing" in LAM and early on with
> >> OPen MPI and decided that it was too hard to meet expected behavior.
> > Some applications rely on this (I know at least two right away, Gamess
> > and Abinit) and they work without problems with Lam/Mpich{1,2} but not
> > with Open MPI. I am *not* arguing that those applications are correct (I
> > agree that this way of passing arguments is ugly, but it's done).
> >
> > I know it's not defined in the standard but I think it's a nice
> > convenient functionality. E.g., setting the LD_LIBRARY_PATH to find
> > libmpi.so in the .bashrc is also a pain if you have multiple (Open) MPIs
> > installed.
>
> LAM does not automatically propogate environment variables -- it's
> behavior is almost *exactly* like Open MPI's. There might be a situation
> where the environment is not quite so scrubbed if a process is started on
> the same node mpirun is executed on, but it's only appearances -- in
> reality, that's the environment that was alive when lamboot was executed.
ok, I might have executed it on the same node (was a while ago).
> With both LAM and Open MPI, there is the -x option to propogate a list of
> environment variables, but that's about it. Neither will push
> LD_LIBRARY_PATH by default (and there are many good reasons for that,
> particularly in heterogeneous situations).
Ah, heterogeneous! Yes, I agree.
Torsten
--
bash$ :(){ :|:&};: - http://www.unixer.de/ -
"I have the result, but I do not yet know how to get it." - Gauss
(1777-1855)
Re: [OMPI devel] Environment forwarding
> This is extremely tricky to do. How do you know which environment > variables to forward (foo in this case) and which not to (hostname). > SLURM has a better chance, since it's linux only and generally only run on > tightly controlled clusters. But there's a whole variety of things that > shouldn't be forwarded and that list differs from OS to OS. > > I believe we toyed around with the "right thing" in LAM and early on with > OPen MPI and decided that it was too hard to meet expected behavior. It's an easy decision for Catamount, so the whole user environment gets sent out to the compute nodes. Anybody care to guess how big that can be when a (goofball) user invokes job launch from within a Makefile? -Ron
Re: [OMPI devel] Environment forwarding
On Mon, 5 Nov 2007, Torsten Hoefler wrote:
On Mon, Nov 05, 2007 at 04:57:19PM -0500, Brian W. Barrett wrote:
This is extremely tricky to do. How do you know which environment
variables to forward (foo in this case) and which not to (hostname).
SLURM has a better chance, since it's linux only and generally only run on
tightly controlled clusters. But there's a whole variety of things that
shouldn't be forwarded and that list differs from OS to OS.
I believe we toyed around with the "right thing" in LAM and early on with
OPen MPI and decided that it was too hard to meet expected behavior.
Some applications rely on this (I know at least two right away, Gamess
and Abinit) and they work without problems with Lam/Mpich{1,2} but not
with Open MPI. I am *not* arguing that those applications are correct (I
agree that this way of passing arguments is ugly, but it's done).
I know it's not defined in the standard but I think it's a nice
convenient functionality. E.g., setting the LD_LIBRARY_PATH to find
libmpi.so in the .bashrc is also a pain if you have multiple (Open) MPIs
installed.
LAM does not automatically propogate environment variables -- it's
behavior is almost *exactly* like Open MPI's. There might be a situation
where the environment is not quite so scrubbed if a process is started on
the same node mpirun is executed on, but it's only appearances -- in
reality, that's the environment that was alive when lamboot was executed.
With both LAM and Open MPI, there is the -x option to propogate a list of
environment variables, but that's about it. Neither will push
LD_LIBRARY_PATH by default (and there are many good reasons for that,
particularly in heterogeneous situations).
Brian
Re: [OMPI devel] Environment forwarding
On Mon, Nov 05, 2007 at 04:57:19PM -0500, Brian W. Barrett wrote:
> This is extremely tricky to do. How do you know which environment
> variables to forward (foo in this case) and which not to (hostname).
> SLURM has a better chance, since it's linux only and generally only run on
> tightly controlled clusters. But there's a whole variety of things that
> shouldn't be forwarded and that list differs from OS to OS.
>
> I believe we toyed around with the "right thing" in LAM and early on with
> OPen MPI and decided that it was too hard to meet expected behavior.
Some applications rely on this (I know at least two right away, Gamess
and Abinit) and they work without problems with Lam/Mpich{1,2} but not
with Open MPI. I am *not* arguing that those applications are correct (I
agree that this way of passing arguments is ugly, but it's done).
I know it's not defined in the standard but I think it's a nice
convenient functionality. E.g., setting the LD_LIBRARY_PATH to find
libmpi.so in the .bashrc is also a pain if you have multiple (Open) MPIs
installed.
Just my two cents,
Torsten
--
bash$ :(){ :|:&};: - http://www.unixer.de/ -
"You should never bet against anything in science at odds of more than
about 10^12 to 1." Ernest Rutherford
Re: [OMPI devel] Environment forwarding
This is extremely tricky to do. How do you know which environment variables to forward (foo in this case) and which not to (hostname). SLURM has a better chance, since it's linux only and generally only run on tightly controlled clusters. But there's a whole variety of things that shouldn't be forwarded and that list differs from OS to OS. I believe we toyed around with the "right thing" in LAM and early on with OPen MPI and decided that it was too hard to meet expected behavior. Brian On Mon, 5 Nov 2007, Tim Prins wrote: Hi, After talking with Torsten today I found something weird. When using the SLURM pls we seem to forward a user's environment, but when using the rsh pls we do not. I.e.: [tprins@odin ~]$ mpirun -np 1 printenv |grep foo [tprins@odin ~]$ export foo=bar [tprins@odin ~]$ mpirun -np 1 printenv |grep foo foo=bar [tprins@odin ~]$ mpirun -np 1 -mca pls rsh printenv |grep foo So my question is which is the expected behavior? I don't think we can do anything about SLURM automatically forwarding the environment, but I think there should be a way to make rsh forward the environment. Perhaps add a flag to mpirun to do this? Thanks, Tim ___ devel mailing list de...@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/devel
[OMPI devel] Environment forwarding
Hi, After talking with Torsten today I found something weird. When using the SLURM pls we seem to forward a user's environment, but when using the rsh pls we do not. I.e.: [tprins@odin ~]$ mpirun -np 1 printenv |grep foo [tprins@odin ~]$ export foo=bar [tprins@odin ~]$ mpirun -np 1 printenv |grep foo foo=bar [tprins@odin ~]$ mpirun -np 1 -mca pls rsh printenv |grep foo So my question is which is the expected behavior? I don't think we can do anything about SLURM automatically forwarding the environment, but I think there should be a way to make rsh forward the environment. Perhaps add a flag to mpirun to do this? Thanks, Tim
Re: [OMPI devel] [OMPI svn] svn:open-mpi r16644
On Nov 5, 2007, at 2:02 PM, Tim Prins wrote: This commit causes a bunch of warnings of the type: runtime/opal_init.c:55:2: warning: #ident is a GCC extension runtime/orte_init.c:92:2: warning: #ident is a GCC extension pinit.c:32:2: warning: #ident is a GCC extension pinit_f.c:26:2: warning: #ident is a GCC extension There's actually a new configure test to figure out which (if any) the compiler supports. Do you know if there's a way to shut up these warnings? Thanks, Tim emall...@osl.iu.edu wrote: Author: emallove Date: 2007-11-02 22:40:22 EDT (Fri, 02 Nov 2007) New Revision: 16644 URL: https://svn.open-mpi.org/trac/ompi/changeset/16644 Log: * Embed ident strings into the Open MPI libraries using one of the following methods (in order of precedence): 1. #pragma ident (e.g., Intel and Sun) 1. #ident (e.g., GCC) 1. static const char ident[] = (all others) By default, the ident string used is the standard Open MPI version string. Only the following libraries will get the embedded version strings (e.g., DSOs will not): * libmpi.so * libmpi_cxx.so * libmpi_f77.so * libopen-pal.so * libopen-rte.so * Added two new configure options: * `--with-package-name="STRING"` (defaults to "Open MPI username@hostname Distribution"). `STRING` is displayed by `ompi_info` next to the "Package" heading. * `--with-ident-string="STRING"` (defaults to the standard Open MPI version string - e.g., X.Y.Zr##). `%VERSION%` will expand to the Open MPI version string if it is supplied to this configure option. Added: trunk/config/ompi_check_ident.m4 Text files modified: trunk/acinclude.m4 | 3 ++- trunk/config/ompi_configure_options.m4 |33 +++ +- trunk/configure.ac |13 ++--- trunk/ompi/include/ompi_config.h.in| 3 +++ trunk/ompi/mpi/c/init.c|10 ++ trunk/ompi/mpi/cxx/mpicxx.cc |12 ++-- trunk/ompi/mpi/f77/init_f.c|10 ++ trunk/ompi/tools/ompi_info/version.cc | 3 +++ trunk/opal/runtime/opal_init.c | 8 trunk/orte/include/orte_config.h.in| 4 trunk/orte/runtime/orte_init.c | 9 + 11 files changed, 101 insertions(+), 7 deletions(-) Modified: trunk/acinclude.m4 = = = = = = = = = = --- trunk/acinclude.m4 (original) +++ trunk/acinclude.m4 2007-11-02 22:40:22 EDT (Fri, 02 Nov 2007) @@ -10,7 +10,7 @@ dnl University of Stuttgart. All rights reserved. dnl Copyright (c) 2004-2005 The Regents of the University of California. dnl All rights reserved. -dnl Copyright (c) 2006 Cisco Systems, Inc. +dnl Copyright (c) 2006-2007 Cisco Systems, Inc. dnl Copyright (c) 2006 Los Alamos National Security, LLC. All rights dnl reserved. dnl Copyright (c) 2007 Sun Microsystems, Inc. All rights reserved. @@ -58,6 +58,7 @@ m4_include(config/ompi_case_sensitive_fs_setup.m4) m4_include(config/ompi_check_broken_qsort.m4) m4_include(config/ompi_check_compiler_works.m4) +m4_include(config/ompi_check_ident.m4) m4_include(config/ompi_check_func_lib.m4) m4_include(config/ompi_check_optflags.m4) m4_include(config/ompi_check_attributes.m4) Added: trunk/config/ompi_check_ident.m4 = = = = = = = = = = --- (empty file) +++ trunk/config/ompi_check_ident.m4 2007-11-02 22:40:22 EDT (Fri, 02 Nov 2007) @@ -0,0 +1,88 @@ +dnl -*- shell-script -*- +dnl +dnl Copyright (c) 2007 Sun Microsystems, Inc. All rights reserved. +dnl $COPYRIGHT$ +dnl +dnl Additional copyrights may follow +dnl +dnl $HEADER$ +dnl +dnl defines: +dnl OMPI_$1_USE_PRAGMA_IDENT +dnl OMPI_$1_USE_IDENT +dnl OMPI_$1_USE_CONST_CHAR_IDENT +dnl + +# OMPI_CHECK_IDENT(compiler-env, compiler-flags, +# file-suffix, lang) Try to compile a source file containing +# a #pragma ident, and determine whether the ident was +# inserted into the resulting object file +# --- +AC_DEFUN([OMPI_CHECK_IDENT], [ +AC_MSG_CHECKING([for $4 ident string support]) + +ompi_pragma_ident_happy=0 +ompi_ident_happy=0 +ompi_static_const_char_happy=0 +_OMPI_CHECK_IDENT( +[$1], [$2], [$3], +[[#]pragma ident], +[ompi_pragma_ident_happy=1 + ompi_message="[#]pragma ident"], +_OMPI_CHECK_IDENT( +[$1], [$2], [$3], +[[#]ident], +[ompi_ident_happy=1 + ompi_message="[#]ident"], +[ompi_static_const_char_happy=1 + ompi_message="static const char[[]]"])) + +AC_DEFINE_UNQUOTED([OMPI_$1_USE_PRAGMA_IDENT], +[$ompi_pragma_ident_happy], [Use #pragma ident strings for $4 files]) +AC_DEFINE_UNQUOTED
[OMPI devel] Multiworld MCA parameter values broken
Hi, Commit 16364 broke things when using multiword mca param values. For instance: mpirun --debug-daemons -mca orte_debug 1 -mca pls rsh -mca pls_rsh_agent "ssh -Y" xterm Will crash and burn, because the value "ssh -Y" is being stored into the argv orted_cmd_line in orterun.c:1506. This is then added to the launch command for the orted: /usr/bin/ssh -Y odin004 PATH=/san/homedirs/tprins/usr/rsl/bin:$PATH ; export PATH ; LD_LIBRARY_PATH=/san/homedirs/tprins/usr/rsl/lib:$LD_LIBRARY_PATH ; export LD_LIBRARY_PATH ; /san/homedirs/tprins/usr/rsl/bin/orted --debug --debug-daemons --name 0.1 --num_procs 2 --vpid_start 0 --nodename odin004 --universe tpr...@odin.cs.indiana.edu:default-universe-27872 --nsreplica "0.0;tcp://129.79.240.100:40907;tcp6://2001:18e8:2:240:2e0:81ff:fe2d:21a0:40908" --gprreplica "0.0;tcp://129.79.240.100:40907;tcp6://2001:18e8:2:240:2e0:81ff:fe2d:21a0:40908" -mca orte_debug 1 -mca pls_rsh_agent ssh -Y -mca mca_base_param_file_path /u/tprins/usr/rsl/share/openmpi/amca-param-sets:/san/homedirs/tprins/rsl/examples -mca mca_base_param_file_path_force /san/homedirs/tprins/rsl/examples Notice that in this command we now have "-mca pls_rsh_agent ssh -Y". So the quotes have been lost, as we die a horrible death. So we need to add the quotes back in somehow, or pass these options differently. I'm not sure what the best way to fix this. Thanks, Tim
Re: [OMPI devel] [OMPI svn] svn:open-mpi r16644
Hi, This commit causes a bunch of warnings of the type: runtime/opal_init.c:55:2: warning: #ident is a GCC extension runtime/orte_init.c:92:2: warning: #ident is a GCC extension pinit.c:32:2: warning: #ident is a GCC extension pinit_f.c:26:2: warning: #ident is a GCC extension Thanks, Tim emall...@osl.iu.edu wrote: Author: emallove Date: 2007-11-02 22:40:22 EDT (Fri, 02 Nov 2007) New Revision: 16644 URL: https://svn.open-mpi.org/trac/ompi/changeset/16644 Log: * Embed ident strings into the Open MPI libraries using one of the following methods (in order of precedence): 1. #pragma ident (e.g., Intel and Sun) 1. #ident (e.g., GCC) 1. static const char ident[] = (all others) By default, the ident string used is the standard Open MPI version string. Only the following libraries will get the embedded version strings (e.g., DSOs will not): * libmpi.so * libmpi_cxx.so * libmpi_f77.so * libopen-pal.so * libopen-rte.so * Added two new configure options: * `--with-package-name="STRING"` (defaults to "Open MPI username@hostname Distribution"). `STRING` is displayed by `ompi_info` next to the "Package" heading. * `--with-ident-string="STRING"` (defaults to the standard Open MPI version string - e.g., X.Y.Zr##). `%VERSION%` will expand to the Open MPI version string if it is supplied to this configure option. Added: trunk/config/ompi_check_ident.m4 Text files modified: trunk/acinclude.m4 | 3 ++- trunk/config/ompi_configure_options.m4 |33 - trunk/configure.ac |13 ++--- trunk/ompi/include/ompi_config.h.in| 3 +++ trunk/ompi/mpi/c/init.c|10 ++ trunk/ompi/mpi/cxx/mpicxx.cc |12 ++-- trunk/ompi/mpi/f77/init_f.c|10 ++ trunk/ompi/tools/ompi_info/version.cc | 3 +++ trunk/opal/runtime/opal_init.c | 8 trunk/orte/include/orte_config.h.in| 4 trunk/orte/runtime/orte_init.c | 9 + 11 files changed, 101 insertions(+), 7 deletions(-) Modified: trunk/acinclude.m4 == --- trunk/acinclude.m4 (original) +++ trunk/acinclude.m4 2007-11-02 22:40:22 EDT (Fri, 02 Nov 2007) @@ -10,7 +10,7 @@ dnl University of Stuttgart. All rights reserved. dnl Copyright (c) 2004-2005 The Regents of the University of California. dnl All rights reserved. -dnl Copyright (c) 2006 Cisco Systems, Inc. +dnl Copyright (c) 2006-2007 Cisco Systems, Inc. dnl Copyright (c) 2006 Los Alamos National Security, LLC. All rights dnl reserved. dnl Copyright (c) 2007 Sun Microsystems, Inc. All rights reserved. @@ -58,6 +58,7 @@ m4_include(config/ompi_case_sensitive_fs_setup.m4) m4_include(config/ompi_check_broken_qsort.m4) m4_include(config/ompi_check_compiler_works.m4) +m4_include(config/ompi_check_ident.m4) m4_include(config/ompi_check_func_lib.m4) m4_include(config/ompi_check_optflags.m4) m4_include(config/ompi_check_attributes.m4) Added: trunk/config/ompi_check_ident.m4 == --- (empty file) +++ trunk/config/ompi_check_ident.m42007-11-02 22:40:22 EDT (Fri, 02 Nov 2007) @@ -0,0 +1,88 @@ +dnl -*- shell-script -*- +dnl +dnl Copyright (c) 2007 Sun Microsystems, Inc. All rights reserved. +dnl $COPYRIGHT$ +dnl +dnl Additional copyrights may follow +dnl +dnl $HEADER$ +dnl +dnl defines: +dnl OMPI_$1_USE_PRAGMA_IDENT +dnl OMPI_$1_USE_IDENT +dnl OMPI_$1_USE_CONST_CHAR_IDENT +dnl + +# OMPI_CHECK_IDENT(compiler-env, compiler-flags, +# file-suffix, lang) Try to compile a source file containing +# a #pragma ident, and determine whether the ident was +# inserted into the resulting object file +# --- +AC_DEFUN([OMPI_CHECK_IDENT], [ +AC_MSG_CHECKING([for $4 ident string support]) + +ompi_pragma_ident_happy=0 +ompi_ident_happy=0 +ompi_static_const_char_happy=0 +_OMPI_CHECK_IDENT( +[$1], [$2], [$3], +[[#]pragma ident], +[ompi_pragma_ident_happy=1 + ompi_message="[#]pragma ident"], +_OMPI_CHECK_IDENT( +[$1], [$2], [$3], +[[#]ident], +[ompi_ident_happy=1 + ompi_message="[#]ident"], +[ompi_static_const_char_happy=1 + ompi_message="static const char[[]]"])) + +AC_DEFINE_UNQUOTED([OMPI_$1_USE_PRAGMA_ID
[OMPI devel] accessors to context id and message id's
Currently in order to do message tracing one either has to rely on some error prone postprocessing of data or replicating some MPI internals up in the PMPI layer. It would help Sun's tools group (and I believe U Dresden also) if Open MPI would create a couple APIs that exoposed the following: 1. PML Message ids used for a request 2. Context id for a specific communicator I could see a couple ways of providing this information. Either by extending the PERUSE probes or creating actual functions that one would pass in a request handle or communicator handle to get the appropriate data back. This is just a thought right now which why this email is not in an RFC format. I wanted to get a feel from the community as to the interest in such APIs and if anyone may have specific issues with us providing such interfaces. If the responses seems positive I will follow this message up with an RFC. thanks, --td
