Thanks for the docs here Brian! Cheers, Chris
On Jul 20, 2012, at 1:35 PM, Brian Foster wrote: > Hey Ricky, > > You can just NFS mount the files to all resource manager nodes like you > mentioned or you can use cas-pge's filestaging support to stage the files > locally using the filemgr... see <fileStagine> element in this file: > https://svn.apache.org/repos/asf/oodt/trunk/pge/src/main/testdata/pge-config.xml > > it will stage all the files in the given metadata fields to the given dir and > will update the metadata fields provided with the new location of the files. > For example in that given pge-config.xml file if: > > InputFiles = /path/to/product.one,/path/to/product.two > > And you have the following configuration (note: force just says that if the > files is already staged there, stage it again): > <fileStaging dir="/my/pge/workspace/input" force="true"> > <stageFiles metadataKey="InputFiles" /> > </fileStaging> > > Then the files would be staged to /my/pge/workspace/input and InputFiles > would now be: > > InputFiles = > /my/pge/workspace/input/product.one,/my/pge/workspace/input/product.two > > -brian > > > On Jul 20, 2012, at 09:35 AM, "Nguyen, Ricky" <[email protected]> wrote: > >> Hi folks, >> >> I was thinking of using Resource Manager to execute my PGE tasks on various >> worker nodes (batch stubs) in hopes of improving the run time of my >> workflows. So my first question is, how will a batch stub obtain files from >> the File Manager? I'm only familiar with the local use case: PGE (without >> resmgr) will just use "$FileLocation/$Filename" returned by filemgr to >> access the file locally. My guess is that all the nodes should mount the >> filemgr archive dir at the same NFS location (as in >> https://cwiki.apache.org/OODT/getting-products-from-a-remote-filemanager.html). >> Or does the PushPull component have a role here? >> >> Thanks, >> Ricky >> >> >> --------------------------------------------------------------------- >> CONFIDENTIALITY NOTICE: This e-mail message, including any attachments, >> is for the sole use of the intended recipient(s) and may contain confidential >> or legally privileged information. Any unauthorized review, use, disclosure >> or distribution is prohibited. If you are not the intended recipient, please >> contact the sender by reply e-mail and destroy all copies of this original >> message. >> >> --------------------------------------------------------------------- >> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
