Have you tried drawing up a Gantt chart that shows the jobs as activities on a time line? I think that would be a revealing action.
Bruce B. On Wed, Apr 18, 2012 at 10:05 AM, Cameron Goodale <sigep...@gmail.com> wrote: > Hey Mike, > > I don't have any formal documentation on the Resource Manager, but I can > try to explain how the Snow Data System team is using it to process large > amounts of MODIS MOD09GA tile data. From there it might be easier to just > ask additional questions since the Components are a lot like legos and you > can swap them around to make several different systems in the end. > > On Snow (Snow Data System Project) we have 5 Servers (1 VM, and 4 Work > Horses with 48 Cores Each). The Workflow Manager and Resource Manager are > installed on the VM and it is sort of like a head node where it does all > the Job Creation and Submission, but it does none of the tile processing. > Here is a list of high-level setups/config that enable us to run the > MOSCAG algorithm across 3 of the 4 Work Horse Machines (the 4th Machine is > used for another processing pipeline, so we will ignore it for now). > > > 1. On the Head Node (VM) we have a Workflow Manager (WM) configured to > Create MODSCAG Workflows. > 2. On the Head Node we also have a Resource Manager (RM) configured to > send jobs out to BatchStubs on the 3 Work Horse Machines. > 3. On the Head Node the WM is configured to USE the RM. This means the > WM will submit jobs to the RM until the RM has reached Capacity, then the > Workflow Manager will queue any additional jobs and wait for the RM to have > additional Capacity by completing a Job. > 4. Our MODSCAG Jobs run in a Quad Core Process, so we have configured > the RM to only submit 10 Jobs to each Work Horse machine (hence using 40 of > the 48 cores). This is unique to the image processing code base, your > mileage (and core usage) may vary. > > So I hope this helps. Again your configuration might be that you have 100 > 4-core machines that you want the RM to use for job processing, or you may > want to stand up BatchStubs on your localhost for testing purposes. At > that point it really is just a matter of getting the configuration setup > and restarting the components. > > If anyone else in the community has anything to add or comment, that would > be great. And Mike if you have any questions at all, don't hesitate to ask. > > Best Regards, > > > Cameron > > > On Tue, Apr 17, 2012 at 9:52 AM, Iwunze, Michael C (GSFC-4700)[NOAA-JPSS] < > michael.iwu...@nasa.gov> wrote: > >> Hello, >> >> I am one of the GRAVITE developers that is new to OODT. Currently >> researching the resource manager component, I saw the user guide online. Do >> you know of any other locations were I can get more information on this >> component as well as more examples and how it can be used? >> >> Thanks >> >> Mike >> > > > > -- > > Sent from a Tin Can attached to a String
