My overall advice for people trying to run Grid Engine on the Amazon
Cloud is this:
(1) If you just want to run Grid Engine in standalone mode on the Amazon
Cloud then you should be using StarCluster
http://web.mit.edu/stardev/cluster/ -- those folks made a fantastic and
free system for elastic SGE clusters on the cloud with working shared
filesystem, MPI configured etc. etc. It's freaking magical. They also
track new AWS features closely and develop support for cool things that
most people would not have time to implement themselves on a small
project -- such as support for running under Spot Instances and within
the odd networking sandbox of the new VPC environments. My company tends
to be the one that runs into StarCluster limitations (such as
inconsistent support for running inside a special VPC zone where a HTTP
proxy was required for internet access) and the developers have been
responsive, friendly and overall nice to interact with.
(2) If you *really* want to do the hybrid cloudbursting thing than I'd
simply say go talk to the nice folks at Univa.com - they already have
far more up to date methods, software and (most importantly) happy
customers using their cloud bursting stuff. This would be the modern and
sustainable route.
(3) If you really don't want to use Univa and you want to bridge a local
SGE cluster into the Amazon cloud than just skip the Hedeby
overhead/complexity and just use Amazon VPC to link your local and
remote subnets together. After that its just one big happy SGE cluster
with a boatload of network/bandwidth limitations affecting some nodes
more than others.
Hedeby was part of a massive project for "resource aware" stuff within
Sun. SGE was just a tiny part. In my mind it has little use or utility
in the context of open source software unless you are going all-in on
all the other hedeby features. Using Hedeby today just to get SGE cloud
adaptors is going to be a stressful exercise involving complex
end-of-life software that is effectively a developmental dead end.
My $.02
-Chris
Dave Love wrote:
Allan Tran<[email protected]> writes:
I'm trying to install hedeby to integrate sge62u5 and amazon cloud but can't
seems to find any good step by step article. All the links seem to be broken
since Oracle kills the open source GE.
Maybe Chris D will chime in with useful guidance. However, as well as
doc at http://wikis.sun.com/display/gridengine62u5/Home (currently
down), you should be able to get everything that was on sunsource via
https://arc.liv.ac.uk/trac/SGE. The source is available via darcs, hg,
and git, or directly under http://arc.liv.ac.uk/repos/darcs/hedeby/&c
(which should work to display the original web pages). Unlike the
actual gridengine source, it hasn't had any love since it was stashed.
I found this
http://wiki.gridengine.info/wiki/index.php/SGE-Hedeby-And-Amazon-EC2#Installing_the_Grid_Engine_Hedeby_Service_Adapter
but
stuck at step Setup SDM Master (sdmadm, etc, where is this sdmadm from? I
know I'm missing a lot of packages but is there a single place I can
download them all.
What I have so far now is a fresh working SGE 62u5 installed with JVM
enabled.
Presumably you need the source from the hedeby, hedeby-ge-adapter, and
hedeby-cloud-adapter repos as above. I can't remember how difficult it
is to build. There are instructions somewhere in the www directory of
the repo if I recall correctly.
_______________________________________________
users mailing list
[email protected]
https://gridengine.org/mailman/listinfo/users
