Am 15.09.2011 um 01:15 schrieb Blosch, Edwin L:
> I would appreciate trying to fix the multi-word argument to
> orte_launch_agent, unless you’ve already fixed it in a later series in which
> case I could update. My workaround with the setuid applied to a copied
> executable doesn’t work for other applications that run on our clusters ( I
> mean their job scripts would also have to be modified similarly).
I played around with `sg` and it looks like having some pitfalls:
$ sg 25000 ls -l
The -l will go to `sg`, even double quotes fo " and ' don't help and there is
no --:
$ sg 2600 ls "'-l'"
gives an error too.
(I was about to suggest an orted wrapper doing:
#!/bin/sh
exec sg 25000 orted ${@}"
but it failed.)
> I think OpenMPI should be able to launch all processes with the same uid and
> gid even when the user has launched the job while using one of his secondary
> groups. Whether it should be a change request or bug (unexpected,
> counter-intuitive), I’ll leave that to your judgment.
Yes, but for now the "username" is used, not the "uid" - right? So it could
have a different uid/gid on the machines, but with the new feature they must be
the same. Okay, in a cluster they are most likely unique across all machines
anyway. But just to note as a side effect.
-- Reuti
> Thank you again for the support
>
>
>
> From: [email protected] [mailto:[email protected]] On
> Behalf Of Ralph Castain
> Sent: Wednesday, September 14, 2011 5:10 PM
> To: Open MPI Users
> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
> created by mpirun?
>
> Hi Ed
>
> I finally had some time to dig around a little. I believe we could make this
> work if I fix the multi-word cmd line param issue. Either "newgrp" or "sg"
> should resolve the problem - both are user-level cmds.
>
> Up to you - let me know if you want me to pursue this.
> Ralph
>
>
> On Sep 14, 2011, at 3:31 PM, Blosch, Edwin L wrote:
>
>
> Thank you - I did pursue this kind of workaround, and it worked, but you’ll
> be happy to know that nothing had to be owned by root.
>
> ASIDE ----
> Just to remind: The job script is a shell script that invokes mpirun; the
> job script itself is run with as the correct user, but the group id may be
> changed to whatever the user requests of the job-scheduling system. I think
> it may not be uncommon to have jobs that request a specific Unix group, for
> many reasons, but in our case the group is an input for the scheduler’s
> prioritization policy.
>
> Outcome: rank 0 runs user:group2 as the user requested, but the launched
> child processes run user:group1 where group 1 is the user’s primary group.
> The peculiarity of this application is that each of the processes writes a
> file to disk, so the resulting group ownership of rank 0 files is group2, but
> the group ownership of all other ranks’ files is group1. That was the
> original problem I’m trying to work around.
> --- END ASIDE
>
> Fortunately for me, there is another peculiarity of this application -- the
> executable gets copied out to /tmp (local space) on each of the hosts to be
> used in the job. We found this helped prevent some crashes during test
> phases where an executable gets overwritten while in use. Definitely a
> special behavior. But as a result of this peculiarity, the mpirun command
> ends up launching the copied executable, and I took advantage of that.
>
> I had the job script do chown user:group2 on the copied executables and then
> chmod 6711, and then I observed that the child processes ran as user:group2,
> same as the rank 0 process, so the files they created had the desired group
> ownership.
>
> I will explore Reuti’s guidance as well.
>
> Thank you
>
>
> From: [email protected] [mailto:[email protected]] On
> Behalf Of Randall Svancara
> Sent: Wednesday, September 14, 2011 3:07 PM
> To: Open MPI Users
> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the processes
> created by mpirun?
>
> You could set the setuid bit on the application and chown it to root?? It is
> about as secure as anything else that has been described thus far. As a
> system admin, I cringe at the thought of anything that would allow something
> to run as someone else, so there would have to be a pretty good
> justification for such unique use case as yours.
>
> Randall
>
> On Wed, Sep 14, 2011 at 12:56 PM, Reuti <[email protected]> wrote:
> Am 14.09.2011 um 19:02 schrieb Blosch, Edwin L:
>
> > Thanks for trying.
> >
> > Do you feel that this is an impossible request without the assistance of
> > some process running as root, for example, as Reuti mentioned, the daemons
> > of a job scheduler? Or are you saying it will just not be as
> > straightforward as calling setgid as you had hoped?
> >
> > Also, do you think there is a way I could make use of the sg command below?
> > Perhaps there is a way to have the rsh/ssh launcher start the application
> > processes with a command like 'sg <group> <executable name>'?
>
> What about a half-tight integration (or call it: classic tight integration),
> i.e. no recompilation necessary?
>
> - setup your mpiexec call in the jobscript to use a plain rsh for the remote
> startup (no path given): –mca plm_rsh_agent rsh
>
> - the PE of SGE needs the argument -catch_rsh in start_proc_args and the
> supplied script in $SGE_ROOT/mpi/startmpi.sh
>
> (SGE will create a symbolic link in $TMPDIR therein [which will be called
> first this way] to the rsh-wrapper in $SGE_ROOT/mpi [pitfall: some
> applications need a -V to be added in the lines woth "qrsh", i.e. "qrsh
> -inherit -V ..." to send all environment variables to the slaves])
>
> - what is your setting of qrsh_daemon/qrsh_command in `qconf -sconf`? This
> will then be used finally to reach the node and should be builtin or point to
> the SGE supplied rsh/rshd (no rshd necessary to install, no rshd is running
> all the time, no rshd will be started by xinet.d or alike)
>
> - like you do already: switch off the built-in SGE starter in your mpiexec
> call: -mca plm_rsh_disable_qrsh 1
>
> -- Reuti
>
> PS: To avoid misunderstandings: you could also set "–mca plm_rsh_agent
> foobar" and in $SGE_ROOT/mpi/startmpi.sh you change it to create a symbolic
> link called "foobar " in $TMPDIR. It's just a name at this stage of startup.
>
>
> > Ed
> >
> >
> > NAME
> > sg - execute command as different group ID
> >
> > SYNOPSIS
> > sg [-] [group [-c ] command]
> >
> > DESCRIPTION
> > The sg command works similar to newgrp but accepts a command. The
> > command will be executed with the /bin/sh shell. With most shells you
> > may run sg from, you need to enclose multi-word commands in quotes.
> > Another difference between newgrp and sg is that some shells treat
> > newgrp specially, replacing themselves with a new instance of a shell
> > that newgrp creates. This doesn't happen with sg, so upon exit from a
> > sg command you are returned to your previous group ID.
> >
> >
> >
> >
> > -----Original Message-----
> > From: [email protected] [mailto:[email protected]] On
> > Behalf Of Ralph Castain
> > Sent: Wednesday, September 14, 2011 11:33 AM
> > To: Open MPI Users
> > Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the
> > processes created by mpirun?
> >
> >
> > On Sep 14, 2011, at 9:39 AM, Blosch, Edwin L wrote:
> >
> >> Thanks, Ralph,
> >>
> >> I get the failure messages, unfortunately:
> >>
> >> setgid FAILED
> >> setgid FAILED
> >> setgid FAILED
> >>
> >> I actually had attempted to call setgid from within the application
> >> previously, which looks similar to what you've done, but it failed. That
> >> was when I initiated the post to the mailing list. My conclusion, a guess
> >> really, was that Linux would not let me setgid from within my program
> >> because I was not root.
> >
> > I was afraid of that - the documentation seemed to indicate that would be
> > the case, but I figured it was worth a quick try. Sorry I can't be of help.
> >
> >
> >>
> >>
> >> -----Original Message-----
> >> From: [email protected] [mailto:[email protected]] On
> >> Behalf Of Ralph Castain
> >> Sent: Wednesday, September 14, 2011 8:15 AM
> >> To: Open MPI Users
> >> Subject: Re: [OMPI users] EXTERNAL: Re: Can you set the gid of the
> >> processes created by mpirun?
> >>
> >> The attached should set the gid of the remote daemons (and their children)
> >> to the gid of mpirun. No cmd line option or anything is required - it will
> >> just always do it.
> >>
> >> Would you mind giving it a try?
> >>
> >> Please let me know if/how it works.
> >>
> >> _______________________________________________
> >> users mailing list
> >> [email protected]
> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> >
> > _______________________________________________
> > users mailing list
> > [email protected]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > _______________________________________________
> > users mailing list
> > [email protected]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> [email protected]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
> --
> Randall Svancara
> http://knowyourlinux.com/
> _______________________________________________
> users mailing list
> [email protected]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> [email protected]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
