I'm trying to figure out a way to define and launch jobs programmatically. The idea is that I am writing a computationally intensive program in some programming language (most typically Java or Scala, but sometimes C, C++, Ruby). I would like to be able, within my programming language, to define some unit of computation that is then launched as an SGE job, with the results returned to the original program.
It appears that something like this sort of thing exists in one form or another (e.g. hadoop, jppf) for various programming languages, but as far as I have been able to determine, these frameworks tend to require their own grid setup. Meaning, I can set up a hadoop or jppf cluster, but that cluster won't interact with my SGE cluster in terms of scheduling jobs. I am aware of DRMAA, but as far as I can tell it isn't what I am looking for. I have successfully used DRMAA to launch (from within Java) SGE jobs defined as external scripts. But what I'm looking for is a mechanism for defining the job itself as Java code to be executed. Or, if I'm writing in C++, as C++ code. I won't be surprised if nothing like this exists, but with all of the work that so many people have done in grid environments over the years, I'm hoping that something like this exists and I just haven't heard about it. Any pointers or suggestions would be greatly appreciated. Cheers, Lane Schwartz _______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
