Hi Valerie,
Is this with or without Radix?
Tom
On 13/11/14 13:10, Mallder, Valerie wrote:
For those of you who are using the pcs-opsui in an operational environment
without getting expired pages please tell me what version you are using, the OS
you are running it on, and the browser you are using.
I need to demonstrate "something" to my project manager to show the usefulness of OODT in
the Jedi instrument science data pipeline here at APL. And right now, I have nothing to
"show" for my last few months of work. If someone can tell me a configuration using opsui
that is working then maybe I can try to mimic that and get something useful up and running. So far,
for me, versions 0.6, 0.7 and the current trunk are showing only expired pages.
Thanks very much!
Val
Sent from my iPhone.
________________________________
From: Mallder, Valerie <[email protected]>
Sent: Tuesday, November 11, 2014 2:55:12 PM
To: [email protected]
Subject: more info for OODT-751 OPSUI Pages constantly expire
Hi Chris,
I know you are working on this, but I wanted to let you know that I tried to
use version 0.6 and am having the same problem in version 0.6. But, I can't
guarantee that I installed version 0.6 correctly so I would like to run this by
you. Based on your email below here's what I did:
1. Downloaded pcs-opsui-0.76.war from:
http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
and saved it in a new folder named $OODT_HOME/bin/opsui
2. Created a script called $OODT_HOME/bin/opsui/runopsui that sets the
following variables (this is output from my script):
Using OODT_BASE:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
Using OODT_HOME:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy
Using OODT_TMPDIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/temp
Using FILEMGR_URL: http://localhost:9000
Using WORKFLOW_URL: http://localhost:9001
Using RESMGR_URL: http://localhost:9002
Using WORKFLOW_HOME:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow
Using RESMGR_HOME:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/resmgr
Using CRAWLER_HOME:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/crawler
Using TOMCAT_HOME:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/tomcat
Using PCS_HOME:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pcs
Using PGE_HOME:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/pge
Using PGE_JOBS_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs
Using FEI_DROP_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/telemetry
Using JEDI_L0_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l0
Using JEDI_L2_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/l2
Using ARCHIVE_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/archive
Using BACKUP_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/met
Using FAILURE_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/failure
Using JEDI_PIPELINE_DIR: /homes/malldva1/working/pipeline
Using SNAPSHOT_DIR:
/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/data/pge/jobs/snapshot
(Then, I followed the steps at the bottom of:
https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI)
3. Downloaded
http://svn.apache.org/repos/asf/oodt/trunk/pcs/opsui/src/main/webapp/META-INF/context.xml
And saved it as $OODT_HOME/bin/opsui/pcs-opsui.xml
4. Edited $OODT_HOME/bin/opsui/pcs-opsui.xml and changed the first line from:
<Context path="/pcs-opsui">
To
<Context path="/pcs-opsui" docBase="[OODT_HOME]/bin/opsui/pcs-opsui-0.6.war">
5. Killed any process with "tomcat" in its name to ensure that tomcat is not
running.
6. Executed the following command to create a symbolic link:
cd $OODT_HOME/bin/opsui/
ln -s pcs-opsui.xml to $TOMCAT_HOME/conf/Catalina/localhost/pcs-opsui.xml
7. Then, in my script that sets the environment variables, I added the
following command to start tomcat.
exec "$OODT_BASE"/tomcat/bin/catalina.sh start
8. Then, I ran my new $OODT_HOME/bin/opsui/runopsui script
9. Then, I started firefox and went to "localhost:8080/pcs-opsui"
The home page comes up, and the PCS Status page comes up, but all other pages are
expired. I should note that the "radix" installation of version 0.7 points me
to localhost:8080/opsui, while step 10 of the instructions from the quick start guide of
the wiki point me to localhost:8080/pcs-opsui. And, no matter which one I go to, the
pages expire.
I also tried clearing out my cache and browser history and restarting firefox
and topcat, etc. and nothing seems to help.
If I have gotten the version 0.6 up and running correctly, I thought you would
be interested to know that I am seeing the error. If I did not get version 0.6
up and running, then I need more hints on how to do it. :)
Thanks,
Val
Valerie A. Mallder
New Horizons Deputy Mission System Engineer
Johns Hopkins University/Applied Physics Laboratory
-----Original Message-----
From: Mattmann, Chris A (3980) [mailto:[email protected]]
Sent: Friday, October 03, 2014 1:07 PM
To: [email protected]
Subject: Re: Success! RE: how to use MetadataBasedFileVersioner properly
Awesome Val! :)
I think you¹re running into this:
https://issues.apache.org/jira/browse/OODT-751
In the meanwhile, try the 0.6 OPSUI, which you can grab from here:
http://repo1.maven.org/maven2/org/apache/oodt/pcs-opsui/0.6/
Grab the WAR file and drop it into your favorite container.
Make sure you have all of these environment variables installed
*before* starting Tomcat or Jetty, etc.:
https://cwiki.apache.org/confluence/display/OODT/Quick+Start+for+PCS+OPSUI
(see steps at bottom and replace 0.5 with 0.6)
I¹m working on a fix for OODT-751, at which point RADIX will be pretty buff.
Next
steps at that point:
1. Release 0.7 and then encourage folks to get started by using the Vagrant
build,
e.g.,
git clone https://github.com/apache/oodt cd vagrant/radix vagrant up
2. Fix OODT-491 and remaining workflow manager issues for Wengine 3. Work
on Streaming OODT API with AMP Stack (via M. Starch et al) and release in 0.8.
4. Conquer and win.
Thanks!
Cheers,
Chris
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398) NASA Jet
Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: [email protected]
WWW: http://sunset.usc.edu/~mattmann/
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++
Adjunct Associate Professor, Computer Science Department University of
Southern California, Los Angeles, CA 90089 USA
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++
-----Original Message-----
From: <Mallder>, Valerie <[email protected]>
Reply-To: "[email protected]" <[email protected]>
Date: Friday, October 3, 2014 at 9:53 AM
To: "[email protected]" <[email protected]>
Subject: Success! RE: how to use MetadataBasedFileVersioner properly
Hi Chris,
Yes, that indeed fixed it! Thanks so much! I now have 18 engineering
files ingested. Whoo hoo!!
Ok, so now, I would like to see what the opsui has to say about my
ingested files. And here comes probably a really stupid question. I am
using Firefox on a Redhat Linux box, and I'm not that familiar with
Firefox and it's settings, so this might be a browser setting issue.
When I startup the opsui and select "File Catalog Browse" it shows that
I have
18 EngineeringFiles. Then, when I select File Catalog Browse from the
strip of options under the logo, I get a message saying the page has
expired. And here's the link that shows up in the address bar:
http://localhost:8080/opsui/?wicket:interface=:7:fmbrowser_link::ILinkL
ist
ener::
And I get the page expired message for all of the options that I select.
Any idea's on this one??
In the meantime though, I will start playing with an action for post
ingestion success that simply makes another copy of all these
engineering files and puts them another folder for Level 0 files.
Thanks,
Val
Valerie A. Mallder
New Horizons Deputy Mission System Engineer Johns Hopkins
University/Applied Physics Laboratory
-----Original Message-----
From: Chris Mattmann [mailto:[email protected]]
Sent: Friday, October 03, 2014 3:03 AM
To: [email protected]
Subject: Re: how to use MetadataBasedFileVersioner properly
You?re almost there Val!
Unfortunately Versioners right now aren?t configurable from product
type policy (would be great to capture this in a JIRA issue, here:
https://issues.apache.org/jira/browse/OODT). If they were, it would
have picked up your <property .. declaration of filePathSpec below.
It?s been on my TODO list for a long time.
Instead I created this:
https://issues.apache.org/jira/browse/OODT-639
So you can amend your definition below (also note you always have to
include /[Filename] at the end to get the filename you want).
<type id="urn:oodt:EngineeringFile" name="EngineeringFile">
<repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
<versioner
class=?org.apache.oodt.cas.filemgr.versioning.ProductTypeMetVersioner?/>
<description>The default product type for any kind of
file.</description>
<metExtractors>
<extractor
class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor">
<configuration>
<!-- you can optionally include the envReplace tag to turn
on/off environment var replacement -->
<property name="nsAware" value="true" />
<property name="elementNs" value="CAS" />
<property name="elements"
value="ProductReceivedTime,ProductName,ProductId" />
</configuration>
</extractor>
</metExtractors>
<metadata>
<keyval>
<key>filePathSpec</key>
<val>/[YearDir]/[DoyDir]/[Filename]</val>
</keyval>
</metadata>
</type>
See if that fixes it!
Cheers,
Chris
------------------------
Chris Mattmann
[email protected]
-----Original Message-----
From: "Mallder, Valerie" <[email protected]>
Reply-To: <[email protected]>
Date: Thursday, October 2, 2014 at 2:56 PM
To: "[email protected]" <[email protected]>
Subject: how to use MetadataBasedFileVersioner properly
Hi All,
I am trying to use the MetadataBasedFileVersioner to store my files
in a subdirectory of the data/archive folder based on the values of
two metadata elements. Has anyone does this before, and if so, can
you give me some hints.
My files need to be organized in subdirectories by year and day of
year.
So, lets say the file name is 'myfile', the year is 2014, and the
doy of year is 002. Then, the end result that I am looking for is to
have the final location of my file be:
'data/archive/ops/eng/2014/002/myfile.
However, in my log file, the INFO messages indicate that the
generated final location reference is 'data/archive/ops/eng/myfile'.
And it doesn't include the year and day of year at all. And the
incorrect location leads to other errors. So I want to solve this
one first. Has anyone tried to do something like this before?
Here is what I have done so far:
Added two new elements to the .met file 'YearDir' and 'DoyDir'.
Added these new elements to the elements.xml file.
Added these new elements to the product in the
product-type-element-map.xml file.
And, I have made changes to the product-types.xml file, but here's
where I am not sure I've done this properly.
Here's what product type definition looks like:
<type id="urn:oodt:EngineeringFile" name="EngineeringFile">
<repository path="file://[OODT_HOME]/data/archive/ops/eng"/>
<versioner
class="org.apache.oodt.cas.filemgr.versioning.MetadataBasedFileVersio
ner
">
<property name="filePathSpec" value="/[YearDir]/[DoyDir]/" />
</versioner>
<description>The default product type for any kind of
file.</description>
<metExtractors>
<extractor
class="org.apache.oodt.cas.filemgr.metadata.extractors.CoreMetExtractor"
<configuration>
<!-- you can optionally include the envReplace tag to turn
on/off environment var replacement -->
<property name="nsAware" value="true" />
<property name="elementNs" value="CAS" />
<property name="elements"
value="ProductReceivedTime,ProductName,ProductId" />
</configuration>
</extractor>
</metExtractors>
<metadata/>
</type>
Thanks in advance for any help or ideas you might have!
Valerie
Valerie A. Mallder
New Horizons Deputy Mission System Engineer The Johns Hopkins
University/Applied Physics Laboratory
11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
240-228-7846 (Office) 410-504-2233 (Blackberry)
--
*Tom Barber* | Technical Director
meteorite bi
*T:* +44 20 8133 3730
*W:* www.meteorite.bi | *Skype:* meteorite.consulting
*A:* Surrey Technology Centre, Surrey Research Park, Guildford, GU2 7YG, UK
