Reuti & William Hay contacted me off-list this morning and told me about "BASH_ENV" - which was not checked in the original version of the patch. Thanks Reuti for the information, and also thanks William for working with everyone and notify us about the bug.
There are way more env variables that can be used by hackers to change system behavior than one can filter. We just filter the common ones (eg. LD_*, Perl's PERLLIB, Python's PYTHONPATH, and now bash's BASH_ENV), but we can't filter anything that has not been invented yet. If security is a big concern, then the prolog, epilog, etc. should be wrapper around a shell script that cleans the environment before starting it. (If compatibility is not a concern, then we could of course do something more extreme - eg. Retui said yesterday on the list that Torque cleans up everything except a few known trusted environment variables before starting the prolog & epilog.) When the patch in Open Grid Scheduler is applied, one can still pass in the "dangerous" environment variables (like: LD_LIBRARY_PATH), but the dangerous env vars are set in the prolog or epilog's environment under a different name (eg. LD_PRELOAD -> SGE_WRAP_LD_PRELOAD). For the actual job, LD_PRELOAD is not affected and is passed to the environment. Lastly, related to the buffer overflow bug in sgepasswd - on Feb 17, Oracle emailed all the Grid Engine implementations about the problem, and William Hay was also notified because that in email discussion thread we were finding a date that we all could release a patch release for the security bugs. Oracle could not share code with us, and it was very hard to ask Oracle to accept changes (and, GPL code is definitely not allowed, so I ended up re-writing the code from scratch to avoid polluting Oracle when we were discussing about the security issues.) The bottomline is that everyone was notified about the 2 problems released yesterday. To those who have never used Oracle software before, Oracle releases CPU (Critical Patch Update) every quarter, and thus the fixes could not just go into Oracle Grid Engine without the shipping vehicle. I added the link to the Oracle Critical Patch Update April 2012 at the bottom of the security page: http://gridscheduler.sourceforge.net/security.html Rayson On Tue, Apr 17, 2012 at 8:23 PM, Reuti <[email protected]> wrote: > Am 17.04.2012 um 23:55 schrieb Ron Chen: > >> Just IMO, IFS & PATH should be set by the script itself. But then it is just >> my opinion _only_. > > I second this and use absolute paths everywhere to be sure. If you know the > next jobnumber you could prepare something in the upcoming $TMPDIR beforehand > otherwise or leave something there for the epilog... > > As said: IMO run scripts only under the user's account himself or use a > "dummy" account (without SGE manager rights) for special purposes. If it's > really necessary to prepare something as root user, I would prefer a suid > binary where LD_PRELOAD is ignored already for the same reason. > > -- Reuti > > >> And there is also LANG, and many other variables that can change the >> behaviour of the script somewhat. >> >> -Ron >> >> >> >> ________________________________ >> From: Reuti <[email protected]> >> To: William Hay <[email protected]> >> Cc: [email protected] >> Sent: Tuesday, April 17, 2012 5:34 PM >> Subject: Re: [gridengine users] Security hole in most versions of Grid Engine >> >> Am 17.04.2012 um 22:57 schrieb William Hay: >> >>> There is a security hole in most current versions of Grid Engine. The >>> problem occurs when using a dynamically linked program as >>> rlogin_daemon, rsh_daemon or qlogin_daemon or when running a >>> dynamically linked binary as prolog or epilog with elevated >>> privileges. This can be exploited by an ordinary user who submits a >>> job using the -v option to set environment variables that control the >>> dynamic linker such as >>> LD_LIBRARY_PATH or LD_PRELOAD on Linux. This allows the user to run >>> arbitrary code with elevated privileges (ie root). >> >> If you run a prolog/epilog script under root account there might be even >> more depending on $PATH or other used (uninitialized) environment variables >> which are used therein. >> >> The best is to run them just as the ordinary user who runs the job anyway. >> >> NB: Don't forget about start/stop_proc_args ;-) >> >> -- Reuti >> >> >>> If you use the builtin daemons and don't have prolog or epilog running >>> with elevated privileges then you should be safe. >>> >>> All the major forks (Son of Grid Engine,Open Grid Scheduler, Univa >>> Grid Engine and Oracle Grid Engine) and several linux distros have >>> prepared patched versions which they should be releasing imminently. >>> >>> If you can't upgrade immediately then it should be possible to >>> protect your system by using a statically linked binary to sanitize >>> the environment. On RedHat and compatible versions of Linux busybox >>> is linked statically and provides an env command that can be used to >>> remove dangerous variables. On Debian and derivatives you need to >>> ensure you have the busybox-static package installed rather than plain >>> busybox. >>> >>> You can then protect your code with something like the following: >>> prolog root@/sbin/busybox env -u BASH_ENV -u >>> LD_LIBRARY_PATH -u LD_PRELOAD -u PERL5OPT -u PERLLIB -u IFS >>> /cm/shared/apps/sge/current/cm/prolog >>> epilog root@/sbin/busybox env -u BASH_ENV -u >>> LD_LIBRARY_PATH -u LD_PRELOAD -u PERL5OPT -u PERLLIB -u IFS >>> /cm/shared/apps/sge/current/cm/epilog >>> rlogin_daemon /sbin/busybox env -u BASH_ENV -u >>> LD_LIBRARY_PATH -u LD_PRELOAD -u PERL5OPT -u PERL5LIB -u PERLLIB -u >>> IFS /cm/shared/apps/sge/assist/bin/qrlogind >>> qlogin_daemon /sbin/busybox env -u BASH_ENV -u >>> LD_LIBRARY_PATH -u LD_PRELOAD -u PERL5OPT -u PERL5LIB -u PERLLIB -u >>> IFS /cm/shared/apps/sge/assist/bin/qlogind >>> rsh_daemon /sbin/busybox env -u BASH_ENV -u >>> LD_LIBRARY_PATH -u LD_PRELOAD -u PERL5OPT -u PERL5LIB -u PERLLIB -u >>> IFS /cm/shared/apps/sge/assist/bin/qrshd >>> >>> The above examples also remove a few other variables that the >>> interpreters we use for our scripts at UCL are sensitive to, >>> >>> Dave Love(of SoGE) has created a small wrapper program that removes >>> the various dynamic linker controlling variables with significantly >>> less verbiage which he should be releasing shortly. >>> >>> >>> >>> William >>> _______________________________________________ >>> users mailing list >>> [email protected] >>> https://gridengine.org/mailman/listinfo/users >> >> >> _______________________________________________ >> users mailing list >> [email protected] >> https://gridengine.org/mailman/listinfo/users >> > > > _______________________________________________ > users mailing list > [email protected] > https://gridengine.org/mailman/listinfo/users _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
