Hi Mike, I'm going to paste some of the conversation that you and I had below earlier b/c I think it's of use to the list in general:
On May 7, 2012, at 3:43 PM, Iwunze, Michael C (GSFC-4700)[NOAA-JPSS] wrote: > > I have two questions, I am able to run the Resource Manager with no issues. I > have some python scripts and possibly some other programs I would like to > run using the Resource Manager. From what I know so far I believe the cas-pge > component needs to be used in conjunction with the Resource Manager and is > used as a wrapper program for running my scripts. Can someone give me more > information on how this can be accomplished or are there any examples to > view? OK, probably the best way to do this would be to actually write a CAS-PGE workflow, and then plug workflow manager into the Resource Manager. There is a pretty good guide for this on the Apache OODT wiki, here: https://cwiki.apache.org/confluence/display/OODT/CAS-PGE+Learn+by+Example Here is the link to the whole wiki: https://cwiki.apache.org/confluence/display/OODT In general the documentation on the Apache OODT wiki is way more up to date than the website which typically only is updated when we make a release. > In addition, I would like to be able to modify or extend the job scheduler > and queue for managing execution of my scripts. Do you know of any examples > of something similar to what I am trying to accomplish? Sure do, and would be happy to share. Check out CAS-PGE above for the running your custom scripts. As for modifying the job scheduler and the queue manager, please find the default implementations of these extensions points, here: http://svn.apache.org/repos/asf/oodt/trunk/resource/src/main/java/org/apache/oodt/cas/resource/scheduler/LRUScheduler.java and here: http://svn.apache.org/repos/asf/oodt/trunk/resource/src/main/java/org/apache/oodt/cas/resource/scheduler/LRUQueueManager.java As for other implementations of them, I have always wanted to create the following: Torque-based Queue implementation that talks to qsub Monitor implementation that talks to Ganglia Batch Mgr implementation that talks to Gexec Batch Mgr implementation that talks to Torque You can see some of these issues here: https://issues.apache.org/jira/browse/OODT-231 And here: https://issues.apache.org/jira/browse/OODT-219 I've been wanting to implement the Ganglia extension since 2005 -- when I was working on the first OCO mission. It's got an easy-to-access, socket/XML based interface that we could write a parser to. Hadoop did this and plugged into Ganglia via its metrics package, so we could probably look at that. HTH and let me know what you think! > > I would also like to be able to utilize the Job Scheduler, Monitor and Job > queue classes that are part of the Resource Manager. I can't find any > examples of how they are used anywhere. And if examples do exist can someone > point me in the right direction or give me more information on this? The Scheduler is the main point of entry, and it's used in the XmlRpcResourceManager class, here: http://s.apache.org/scU Scheduler calls Monitor, as well as JobQueue, and is the nexus of interaction there. HTH get you started! Cheers, Chris ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
