Oi Chris, I sent Val a fix! :P On 21/11/14 14:46, Mattmann, Chris A (3980) wrote: > Hi Val, > > I haven’t forgot about this - just haven’t had a chance to investigate. > Hope to have some time in the next week. > > Cheers, > Chris > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > > -----Original Message----- > From: <Mallder>, Valerie <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Thursday, November 13, 2014 at 2:10 PM > To: "[email protected]" <[email protected]> > Subject: RE: more info for OODT-751 OPSUI Pages constantly expire > >> For those of you who are using the pcs-opsui in an operational >> environment without getting expired pages please tell me what version you >> are using, the OS you are running it on, and the browser you are using. >> >> I need to demonstrate "something" to my project manager to show the >> usefulness of OODT in the Jedi instrument science data pipeline here at >> APL. And right now, I have nothing to "show" for my last few months of >> work. If someone can tell me a configuration using opsui that is working >> then maybe I can try to mimic that and get something useful up and >> running. So far, for me, versions 0.6, 0.7 and the current trunk are >> showing only expired pages. >> >> Thanks very much! >> Val >> >> >> >> Sent from my iPhone. >> ________________________________ >> From: Mallder, Valerie <[email protected]> >> Sent: Tuesday, November 11, 2014 2:55:12 PM >> To: [email protected] >> Subject: more info for OODT-751 OPSUI Pages constantly expire >> >> Hi Chris, >> >> I know you are working on this, but I wanted to let you know that I tried >> to use version 0.6 and am having the same problem in version 0.6. But, I >> can't guarantee that I installed version 0.6 correctly so I would like to >> run this by you. Based on your email below here's what I did: >> >> 1. Downloaded pcs-opsui-0.76.war from: >> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/ >> and saved it in a new folder named $OODT_HOME/bin/opsui >> >> 2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the >> following variables (this is output from my script): >> Using OODT_BASE: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy >> Using OODT_HOME: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy >> Using OODT_TMPDIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp >> Using FILEMGR_URL: http://localhost:9000 >> Using WORKFLOW_URL: http://localhost:9001 >> Using RESMGR_URL: http://localhost:9002 >> Using WORKFLOW_HOME: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow >> Using RESMGR_HOME: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr >> Using CRAWLER_HOME: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler >> Using TOMCAT_HOME: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat >> Using PCS_HOME: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs >> Using PGE_HOME: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge >> Using PGE_JOBS_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs >> Using FEI_DROP_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/telemetr >> y >> Using JEDI_L0_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0 >> Using JEDI_L2_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2 >> Using ARCHIVE_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive >> Using BACKUP_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met >> Using FAILURE_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/failure >> Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline >> Using SNAPSHOT_DIR: >> /homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs >> /snapshot >> >> (Then, I followed the steps at the bottom of: >> https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI >> ) >> >> 3. Downloaded >> http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META- >> INF/context.xml >> And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml >> >> 4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first line >> from: >> <Context path="/pcs-opsui"> >> To >> <Context path="/pcs-opsui" >> docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war"> >> >> 5. Killed any process with "tomcat" in its name to ensure that tomcat is >> not running. >> >> 6. Executed the following command to create a symbolic link: >> cd $OODT_HOME/bin/opsui/ >> ln -s pcs-opsui.xml to $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml >> >> 7. Then, in my script that sets the environment variables, I added the >> following command to start tomcat. >> exec "$OODT_BASE"/tomcat/bin/catalina.sh start >> >> 8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script >> >> 9. Then, I started firefox and went to "localhost:8080/pcs-opsui" >> >> >> >> The home page comes up, and the PCS Status page comes up, but all other >> pages are expired. I should note that the "radix" installation of >> version 0.7 points me to localhost:8080/opsui, while step 10 of the >> instructions from the quick start guide of the wiki point me to >> localhost:8080/pcs-opsui. And, no matter which one I go to, the pages >> expire. >> >> I also tried clearing out my cache and browser history and restarting >> firefox and topcat, etc. and nothing seems to help. >> >> If I have gotten the version 0.6 up and running correctly, I thought you >> would be interested to know that I am seeing the error. If I did not get >> version 0.6 up and running, then I need more hints on how to do it. :) >> >> Thanks, >> Val >> >> >> >> >> >> >> >> >> >> Valerie A. Mallder >> New Horizons Deputy Mission System Engineer >> Johns Hopkins University/Applied Physics Laboratory >> >> >>> -----Original Message----- >>> From: Mattmann, Chris A (3980) [mailto:[email protected]] >>> Sent: Friday, October 03, 2014 1:07 PM >>> To: [email protected] >>> Subject: Re: Success! RE: how to use MetadataBasedFileVersioner properly >>> >>> Awesome Val! :) >>> >>> I think you¹re running into this: >>> >>> https://issues.apache.org/jira/browse/OODT-751 >>> >>> >>> In the meanwhile, try the 0.6 OPSUI, which you can grab from here: >>> >>> http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/ >>> >>> >>> Grab the WAR file and drop it into your favorite container. >>> Make sure you have all of these environment variables installed >>> *before* starting Tomcat or Jetty, etc.: >>> >>> >>> https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSU >>> I >>> >>> >>> (see steps at bottom and replace 0.5 with 0.6) >>> >>> I¹m working on a fix for OODT-751, at which point RADIX will be pretty >>> buff. Next >>> steps at that point: >>> >>> 1. Release 0.7 and then encourage folks to get started by using the >>> Vagrant build, >>> e.g., >>> >>> git clone https://github.com/apache/oodt cd vagrant/radix vagrant up >>> >>> 2. Fix OODT-491 and remaining workflow manager issues for Wengine 3. >>> Work >>> on Streaming OODT API with AMP Stack (via M. Starch et al) and release >>> in 0.8. >>> 4. Conquer and win. >>> >>> Thanks! >>> >>> Cheers, >>> Chris >>> >>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> +++++ >>> Chris Mattmann, Ph.D. >>> Chief Architect >>> Instrument Software and Science Data Systems Section (398) NASA Jet >>> Propulsion Laboratory Pasadena, CA 91109 USA >>> Office: 168-519, Mailstop: 168-527 >>> Email: [email protected] >>> WWW: http://sunset.usc.edu/~mattmann/ >>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> +++++ >>> Adjunct Associate Professor, Computer Science Department University of >>> Southern California, Los Angeles, CA 90089 USA >>> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> +++++ >>> >>> >>> >>> >>> >>> >>> -----Original Message----- >>> From: <Mallder>, Valerie <[email protected]> >>> Reply-To: "[email protected]" <[email protected]> >>> Date: Friday, October 3, 2014 at 9:53 AM >>> To: "[email protected]" <[email protected]> >>> Subject: Success! RE: how to use MetadataBasedFileVersioner properly >>> >>>> Hi Chris, >>>> >>>> Yes, that indeed fixed it! Thanks so much! I now have 18 engineering >>>> files ingested. Whoo hoo!! >>>> >>>> Ok, so now, I would like to see what the opsui has to say about my >>>> ingested files. And here comes probably a really stupid question. I am >>>> using Firefox on a Redhat Linux box, and I'm not that familiar with >>>> Firefox and it's settings, so this might be a browser setting issue. >>>> When I startup the opsui and select "File Catalog Browse" it shows that >>>> I have >>>> 18 EngineeringFiles. Then, when I select File Catalog Browse from the >>>> strip of options under the logo, I get a message saying the page has >>>> expired. And here's the link that shows up in the address bar: >>>> http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkL >>>> ist >>>> ener:: >>>> >>>> And I get the page expired message for all of the options that I >>> select. >>>> Any idea's on this one?? >>>> >>>> In the meantime though, I will start playing with an action for post >>>> ingestion success that simply makes another copy of all these >>>> engineering files and puts them another folder for Level 0 files. >>>> >>>> Thanks, >>>> Val >>>> >>>> >>>> >>>> >>>> Valerie A. Mallder >>>> New Horizons Deputy Mission System Engineer Johns Hopkins >>>> University/Applied Physics Laboratory >>>> >>>> >>>>> -----Original Message----- >>>>> From: Chris Mattmann [mailto:[email protected]] >>>>> Sent: Friday, October 03, 2014 3:03 AM >>>>> To: [email protected] >>>>> Subject: Re: how to use MetadataBasedFileVersioner properly >>>>> >>>>> You?re almost there Val! >>>>> >>>>> Unfortunately Versioners right now aren?t configurable from product >>>>> type policy (would be great to capture this in a JIRA issue, here: >>>>> https://issues.apache.org/jira/browse/OODT). If they were, it would >>>>> have picked up your <property .. declaration of filePathSpec below. >>>>> It?s been on my TODO list for a long time. >>>>> >>>>> Instead I created this: >>>>> >>>>> https://issues.apache.org/jira/browse/OODT-639 >>>>> >>>>> >>>>> So you can amend your definition below (also note you always have to >>>>> include /[Filename] at the end to get the filename you want). >>>>> >>>>> <type id="urn:oodt:EngineeringFile" name="EngineeringFile"> >>>>> <repository path="file://[OODT_HOME]/data/archive/ops/eng"/> >>>>> <versioner >>>>> >>> class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/> >>>>> <description>The default product type for any kind of >>>>> file.</description> >>>>> <metExtractors> >>>>> <extractor >>>>> >>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor >>>>> "> >>>>> <configuration> >>>>> <!-- you can optionally include the envReplace tag to turn >>>>> on/off environment var replacement --> >>>>> <property name="nsAware" value="true" /> >>>>> <property name="elementNs" value="CAS" /> >>>>> <property name="elements" >>>>> value="ProductReceivedTime,ProductName,ProductId" /> >>>>> </configuration> >>>>> </extractor> >>>>> </metExtractors> >>>>> <metadata> >>>>> <keyval> >>>>> <key>filePathSpec</key> >>>>> <val>/[YearDir]/[DoyDir]/[Filename]</val> >>>>> </keyval> >>>>> >>>>> </metadata> >>>>> </type> >>>>> >>>>> >>>>> See if that fixes it! >>>>> >>>>> Cheers, >>>>> Chris >>>>> >>>>> ------------------------ >>>>> Chris Mattmann >>>>> [email protected] >>>>> >>>>> >>>>> >>>>> >>>>> -----Original Message----- >>>>> From: "Mallder, Valerie" <[email protected]> >>>>> Reply-To: <[email protected]> >>>>> Date: Thursday, October 2, 2014 at 2:56 PM >>>>> To: "[email protected]" <[email protected]> >>>>> Subject: how to use MetadataBasedFileVersioner properly >>>>> >>>>>> Hi All, >>>>>> >>>>>> I am trying to use the MetadataBasedFileVersioner to store my files >>>>>> in a subdirectory of the data/archive folder based on the values of >>>>>> two metadata elements. Has anyone does this before, and if so, can >>>>>> you give me some hints. >>>>>> >>>>>> My files need to be organized in subdirectories by year and day of >>>>> year. >>>>>> So, lets say the file name is 'myfile', the year is 2014, and the >>>>>> doy of year is 002. Then, the end result that I am looking for is to >>>>>> have the final location of my file be: >>>>> 'data/archive/ops/eng/2014/002/myfile. >>>>>> However, in my log file, the INFO messages indicate that the >>>>>> generated final location reference is 'data/archive/ops/eng/myfile'. >>>>>> And it doesn't include the year and day of year at all. And the >>>>>> incorrect location leads to other errors. So I want to solve this >>>>>> one first. Has anyone tried to do something like this before? >>>>>> >>>>>> Here is what I have done so far: >>>>>> >>>>>> Added two new elements to the .met file 'YearDir' and 'DoyDir'. >>>>>> Added these new elements to the elements.xml file. >>>>>> Added these new elements to the product in the >>>>>> product-type-element-map.xml file. >>>>>> And, I have made changes to the product-types.xml file, but here's >>>>>> where I am not sure I've done this properly. >>>>>> >>>>>> Here's what product type definition looks like: >>>>>> <type id="urn:oodt:EngineeringFile" name="EngineeringFile"> >>>>>> <repository path="file://[OODT_HOME]/data/archive/ops/eng"/> >>>>>> <versioner >>>>>> class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersio >>>>>> ner >>>>>> "> >>>>>> <property name="filePathSpec" >>> value="/[YearDir]/[DoyDir]/" /> >>>>>> </versioner> >>>>>> <description>The default product type for any kind of >>>>>> file.</description> >>>>>> <metExtractors> >>>>>> <extractor >>>>>> class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtracto >>>>>> r" >>>>>> <configuration> >>>>>> <!-- you can optionally include the envReplace tag to turn >>>>>> on/off environment var replacement --> >>>>>> <property name="nsAware" value="true" /> >>>>>> <property name="elementNs" value="CAS" /> >>>>>> <property name="elements" >>>>>> value="ProductReceivedTime,ProductName,ProductId" /> >>>>>> </configuration> >>>>>> </extractor> >>>>>> </metExtractors> >>>>>> <metadata/> >>>>>> </type> >>>>>> >>>>>> Thanks in advance for any help or ideas you might have! >>>>>> >>>>>> Valerie >>>>>> >>>>>> >>>>>> >>>>>> Valerie A. Mallder >>>>>> >>>>>> New Horizons Deputy Mission System Engineer The Johns Hopkins >>>>>> University/Applied Physics Laboratory >>>>>> 11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723 >>>>>> 240-228-7846 (Office) 410-504-2233 (Blackberry) >>>>>>
-- *Tom Barber* | Technical Director meteorite bi *T:* +44 20 8133 3730 *W:* www.meteorite.bi | *Skype:* meteorite.consulting *A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK
