Hi Tyler, Thanks for writing! I have heard of DRAT (Chris pointed me to it for a examples of a running PGE) but I have not used it and I don't really know what it does. If you think it could help me figure out something, and there is a good source of documentation to help me use it, I will be happy to try it.
Do you happen to know what "GANGLIA" is and what default "url" I should set the GANGLIA_URL environment too? I see this variable in the http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-INF/context.xml file and this is the first I have seen of it. I've been searching the mailing list archives for any information I can find on opsui, and I found out that there are several ways to browse for products and files. All of which are greek to me. So, I have no idea where to start in trying to use any one of them. While I am familiar with html, css, php, mysql, and a little bit of javascript for building websites, I have never used tomcat and I don't know what a "web app", a "wicket web app" or a "REST service" are. (maybe I do, but by a different name?) So, I'm a bit at a disadvantage here. Thanks, Val Valerie A. Mallder New Horizons Deputy Mission System Engineer Johns Hopkins University/Applied Physics Laboratory > -----Original Message----- > From: Tyler Palsulich [mailto:[email protected]] > Sent: Thursday, November 13, 2014 11:36 AM > To: dev > Subject: RE: more info for OODT-751 OPSUI Pages constantly expire > > Have you tried installing and running DRAT? > > Tyler > On Nov 13, 2014 8:11 AM, "Mallder, Valerie" <[email protected]> > wrote: > > > For those of you who are using the pcs-opsui in an operational > > environment without getting expired pages please tell me what version > > you are using, the OS you are running it on, and the browser you are using. > > > > I need to demonstrate "something" to my project manager to show the > > usefulness of OODT in the Jedi instrument science data pipeline here > > at APL. And right now, I have nothing to "show" for my last few months > > of work. If someone can tell me a configuration using opsui that is > > working then maybe I can try to mimic that and get something useful up and > running. > > So far, for me, versions 0.6, 0.7 and the current trunk are showing > > only expired pages. > > > > Thanks very much! > > Val > > > > > > > > Sent from my iPhone. > > ________________________________ > > From: Mallder, Valerie <[email protected]> > > Sent: Tuesday, November 11, 2014 2:55:12 PM > > To: [email protected] > > Subject: more info for OODT-751 OPSUI Pages constantly expire > > > > Hi Chris, > > > > I know you are working on this, but I wanted to let you know that I > > tried to use version 0.6 and am having the same problem in version > > 0.6. But, I can't guarantee that I installed version 0.6 correctly so > > I would like to run this by you. Based on your email below here's what I > > did: > > > > 1. Downloaded pcs-opsui-0.76.war from: > > http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/ > > and saved it in a new folder named $OODT_HOME/bin/opsui > > > > 2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the > > following variables (this is output from my script): > > Using OODT_BASE: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy > > Using OODT_HOME: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy > > Using OODT_TMPDIR: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp > > Using FILEMGR_URL: http://localhost:9000 > > Using WORKFLOW_URL: http://localhost:9001 > > Using RESMGR_URL: http://localhost:9002 > > Using WORKFLOW_HOME: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow > > Using RESMGR_HOME: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr > > Using CRAWLER_HOME: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler > > Using TOMCAT_HOME: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat > > Using PCS_HOME: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs > > Using PGE_HOME: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge > > Using PGE_JOBS_DIR: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/ > > jobs > > Using FEI_DROP_DIR: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/tele > > metry > > Using JEDI_L0_DIR: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0 > > Using JEDI_L2_DIR: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2 > > Using ARCHIVE_DIR: > > > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/arch > > ive > > Using BACKUP_DIR: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met > > Using FAILURE_DIR: > > > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/fail > > ure Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline Using > > SNAPSHOT_DIR: > > /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/ > > jobs/snapshot > > > > (Then, I followed the steps at the bottom of: > > https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+O > > PSUI > > ) > > > > 3. Downloaded > > http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/M > > ETA-INF/context.xml And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml > > > > 4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first > > line > > from: > > <Context path="/pcs-opsui"> > > To > > <Context path="/pcs-opsui" > > docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war"> > > > > 5. Killed any process with "tomcat" in its name to ensure that tomcat > > is not running. > > > > 6. Executed the following command to create a symbolic link: > > cd $OODT_HOME/bin/opsui/ > > ln -s pcs-opsui.xml to > > $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml > > > > 7. Then, in my script that sets the environment variables, I added the > > following command to start tomcat. > > exec "$OODT_BASE"/tomcat/bin/catalina.sh start > > > > 8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script > > > > 9. Then, I started firefox and went to "localhost:8080/pcs-opsui" > > > > > > > > The home page comes up, and the PCS Status page comes up, but all > > other pages are expired. I should note that the "radix" installation > > of version > > 0.7 points me to localhost:8080/opsui, while step 10 of the > > instructions from the quick start guide of the wiki point me to > > localhost:8080/pcs-opsui. And, no matter which one I go to, the pages > > expire. > > > > I also tried clearing out my cache and browser history and restarting > > firefox and topcat, etc. and nothing seems to help. > > > > If I have gotten the version 0.6 up and running correctly, I thought > > you would be interested to know that I am seeing the error. If I did > > not get version 0.6 up and running, then I need more hints on how to > > do it. :) > > > > Thanks, > > Val > > > > > > > > > > > > > > > > > > > > Valerie A. Mallder > > New Horizons Deputy Mission System Engineer Johns Hopkins > > University/Applied Physics Laboratory > > > > > > > -----Original Message----- > > > From: Mattmann, Chris A (3980) > > > [mailto:[email protected]] > > > Sent: Friday, October 03, 2014 1:07 PM > > > To: [email protected] > > > Subject: Re: Success! RE: how to use MetadataBasedFileVersioner > > > properly > > > > > > Awesome Val! :) > > > > > > I think you¹re running into this: > > > > > > https://issues.apache.org/jira/browse/OODT-751 > > > > > > > > > In the meanwhile, try the 0.6 OPSUI, which you can grab from here: > > > > > > http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/ > > > > > > > > > Grab the WAR file and drop it into your favorite container. > > > Make sure you have all of these environment variables installed > > > *before* starting Tomcat or Jetty, etc.: > > > > > > > > https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+O > > PSUI > > > > > > > > > (see steps at bottom and replace 0.5 with 0.6) > > > > > > I¹m working on a fix for OODT-751, at which point RADIX will be > > > pretty > > buff. Next > > > steps at that point: > > > > > > 1. Release 0.7 and then encourage folks to get started by using the > > Vagrant build, > > > e.g., > > > > > > git clone https://github.com/apache/oodt cd vagrant/radix vagrant up > > > > > > 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3. > > > Work on Streaming OODT API with AMP Stack (via M. Starch et al) and > > > release > > in 0.8. > > > 4. Conquer and win. > > > > > > Thanks! > > > > > > Cheers, > > > Chris > > > > > > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > +++++ > > > Chris Mattmann, Ph.D. > > > Chief Architect > > > Instrument Software and Science Data Systems Section (398) NASA Jet > > > Propulsion Laboratory Pasadena, CA 91109 USA > > > Office: 168-519, Mailstop: 168-527 > > > Email: [email protected] > > > WWW: http://sunset.usc.edu/~mattmann/ > > > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > +++++ > > > Adjunct Associate Professor, Computer Science Department University > > > of Southern California, Los Angeles, CA 90089 USA > > > > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > +++++ > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > From: <Mallder>, Valerie <[email protected]> > > > Reply-To: "[email protected]" <[email protected]> > > > Date: Friday, October 3, 2014 at 9:53 AM > > > To: "[email protected]" <[email protected]> > > > Subject: Success! RE: how to use MetadataBasedFileVersioner > > > properly > > > > > > >Hi Chris, > > > > > > > >Yes, that indeed fixed it! Thanks so much! I now have 18 > > > >engineering files ingested. Whoo hoo!! > > > > > > > >Ok, so now, I would like to see what the opsui has to say about my > > > >ingested files. And here comes probably a really stupid question. > > > >I am using Firefox on a Redhat Linux box, and I'm not that familiar > > > >with Firefox and it's settings, so this might be a browser setting issue. > > > >When I startup the opsui and select "File Catalog Browse" it shows > > > >that I have > > > >18 EngineeringFiles. Then, when I select File Catalog Browse from > > > >the strip of options under the logo, I get a message saying the > > > >page has expired. And here's the link that shows up in the address bar: > > > >http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::IL > > > >inkL > > > >ist > > > >ener:: > > > > > > > >And I get the page expired message for all of the options that I select. > > > >Any idea's on this one?? > > > > > > > >In the meantime though, I will start playing with an action for > > > >post ingestion success that simply makes another copy of all these > > > >engineering files and puts them another folder for Level 0 files. > > > > > > > >Thanks, > > > >Val > > > > > > > > > > > > > > > > > > > >Valerie A. Mallder > > > >New Horizons Deputy Mission System Engineer Johns Hopkins > > > >University/Applied Physics Laboratory > > > > > > > > > > > >> -----Original Message----- > > > >> From: Chris Mattmann [mailto:[email protected]] > > > >> Sent: Friday, October 03, 2014 3:03 AM > > > >> To: [email protected] > > > >> Subject: Re: how to use MetadataBasedFileVersioner properly > > > >> > > > >> You?re almost there Val! > > > >> > > > >> Unfortunately Versioners right now aren?t configurable from > > > >>product type policy (would be great to capture this in a JIRA issue, > > > >>here: > > > >> https://issues.apache.org/jira/browse/OODT). If they were, it > > > >>would have picked up your <property .. declaration of filePathSpec > > > >>below. > > > >> It?s been on my TODO list for a long time. > > > >> > > > >> Instead I created this: > > > >> > > > >> https://issues.apache.org/jira/browse/OODT-639 > > > >> > > > >> > > > >> So you can amend your definition below (also note you always have > > > >>to include /[Filename] at the end to get the filename you want). > > > >> > > > >> <type id="urn:oodt:EngineeringFile" name="EngineeringFile"> > > > >> <repository path="file://[OODT_HOME]/data/archive/ops/eng"/> > > > >> <versioner > > > >> > > class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner? > > /> > > > >> <description>The default product type for any kind of > > > >>file.</description> > > > >> <metExtractors> > > > >> <extractor > > > >> > > > > > >>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtrac > > >>tor"> > > > >> <configuration> > > > >> <!-- you can optionally include the envReplace tag to > > > >>turn on/off environment var replacement --> > > > >> <property name="nsAware" value="true" /> > > > >> <property name="elementNs" value="CAS" /> > > > >> <property name="elements" > > > >> value="ProductReceivedTime,ProductName,ProductId" /> > > > >> </configuration> > > > >> </extractor> > > > >> </metExtractors> > > > >> <metadata> > > > >> <keyval> > > > >> <key>filePathSpec</key> > > > >> <val>/[YearDir]/[DoyDir]/[Filename]</val> > > > >> </keyval> > > > >> > > > >> </metadata> > > > >> </type> > > > >> > > > >> > > > >> See if that fixes it! > > > >> > > > >> Cheers, > > > >> Chris > > > >> > > > >> ------------------------ > > > >> Chris Mattmann > > > >> [email protected] > > > >> > > > >> > > > >> > > > >> > > > >> -----Original Message----- > > > >> From: "Mallder, Valerie" <[email protected]> > > > >> Reply-To: <[email protected]> > > > >> Date: Thursday, October 2, 2014 at 2:56 PM > > > >> To: "[email protected]" <[email protected]> > > > >> Subject: how to use MetadataBasedFileVersioner properly > > > >> > > > >> >Hi All, > > > >> > > > > >> >I am trying to use the MetadataBasedFileVersioner to store my > > > >> >files in a subdirectory of the data/archive folder based on the > > > >> >values of two metadata elements. Has anyone does this before, > > > >> >and if so, can you give me some hints. > > > >> > > > > >> >My files need to be organized in subdirectories by year and day > > > >> >of > > > >>year. > > > >> >So, lets say the file name is 'myfile', the year is 2014, and > > > >> >the doy of year is 002. Then, the end result that I am looking > > > >> >for is to have the final location of my file be: > > > >>'data/archive/ops/eng/2014/002/myfile. > > > >> > > > > >> >However, in my log file, the INFO messages indicate that the > > > >> >generated final location reference is 'data/archive/ops/eng/myfile'. > > > >> >And it doesn't include the year and day of year at all. And the > > > >> >incorrect location leads to other errors. So I want to solve > > > >> >this one first. Has anyone tried to do something like this before? > > > >> > > > > >> >Here is what I have done so far: > > > >> > > > > >> >Added two new elements to the .met file 'YearDir' and 'DoyDir'. > > > >> >Added these new elements to the elements.xml file. > > > >> >Added these new elements to the product in the > > > >> >product-type-element-map.xml file. > > > >> >And, I have made changes to the product-types.xml file, but > > > >> >here's where I am not sure I've done this properly. > > > >> > > > > >> >Here's what product type definition looks like: > > > >> > <type id="urn:oodt:EngineeringFile" name="EngineeringFile"> > > > >> > <repository path="file://[OODT_HOME]/data/archive/ops/eng"/> > > > >> > <versioner > > > >> > > > >>>class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVe > > > >>>rsio > > > >>>ner > > > >>>"> > > > >> > <property name="filePathSpec" value="/[YearDir]/[DoyDir]/" > > /> > > > >> > </versioner> > > > >> > <description>The default product type for any kind of > > > >> >file.</description> > > > >> > <metExtractors> > > > >> > <extractor > > > >> > > > > > >>>class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor" > > > >>>> > > > >> > <configuration> > > > >> > <!-- you can optionally include the envReplace tag to > > > >> >turn on/off environment var replacement --> > > > >> > <property name="nsAware" value="true" /> > > > >> > <property name="elementNs" value="CAS" /> > > > >> > <property name="elements" > > > >> >value="ProductReceivedTime,ProductName,ProductId" /> > > > >> > </configuration> > > > >> > </extractor> > > > >> > </metExtractors> > > > >> > <metadata/> > > > >> > </type> > > > >> > > > > >> >Thanks in advance for any help or ideas you might have! > > > >> > > > > >> >Valerie > > > >> > > > > >> > > > > >> > > > > >> >Valerie A. Mallder > > > >> > > > > >> >New Horizons Deputy Mission System Engineer The Johns Hopkins > > > >> >University/Applied Physics Laboratory > > > >> >11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723 > > > >> >240-228-7846 (Office) 410-504-2233 (Blackberry) > > > >> > > > > >> > > > > > > > >
