Hello,
I'm stuck again L This time I'm stuck trying to start my crawler as a task
using the workflow manager. I am not using a PGE task right now. I'm just
trying to do something simple with the workflow manager, filemgr, and crawler.
I have read all of the documentation that is available on the workflow manager
and have tried to piece together a setup based on the examples, but, things
seem to be working differently now and the documentation hasn't caught up,
which is totally understandable and not a criticism. Just want you to know
that I try to do my due diligence before bothering anyone for help.
I am not running the resource manager, and I have commented out setting the
resource manager url in the workflow.properties file so that workflow manager
will execute the job locally.
I am sending workflow manager an event (via the command line using wmgr-client)
called "startJediPipeline". Workflow manager receives the event, and retrieves
my workflow from the repository and tries to execute the first (and only) task,
and then it crashes. My task is an external script (the crawler_launcher
script) and I need to pass several arguments to it. I've spent all day trying
to figure out how to pass arguments to the and ExternScriptTaskInstance, but
there are no examples of doing this, so I had to wing it. I tried putting the
arguments in the task configuration properties. That didn't work. So I tried
putting the arguments in the metadata properties, and that hasn't worked. So,
your suggestions are welcome! Thanks so much. Here's the error log, And
contents of my tasks.xml file follow it at the end.
Workflow Manager started PID file
(/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow/run/cas.workflow.pid).
Starting OODT File Manager [ Successful ]
Starting OODT Resource Manager [ Failed ]
Starting OODT Workflow Manager [ Successful ]
slothrop:{~/project/jedi/users/jedi-pipeline/oodt-deploy/bin} Oct 06, 2014
5:48:30 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager
loadProperties
INFO: Loading Workflow Manager Configuration Properties from:
[/homes/malldva1/project/jedi/users/jedi-pipeline/oodt-deploy/workflow/etc/workflow.properties]
Oct 06, 2014 5:48:30 PM
org.apache.oodt.cas.workflow.engine.ThreadPoolWorkflowEngineFactory getResmgrUrl
INFO: No Resource Manager URL provided or malformed URL: executing jobs
locally. URL: [null]
Oct 06, 2014 5:48:30 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init>
INFO: Workflow Manager started by malldva1
Oct 06, 2014 5:48:41 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Received event: startJediPipeline
Oct 06, 2014 5:48:41 PM
org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent
INFO: WorkflowManager: Workflow Jedi Pipeline Workflow retrieved for event
startJediPipeline
Oct 06, 2014 5:48:41 PM
org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
checkTaskRequiredMetadata
INFO: Task: [Crawler Task] has no required metadata fields
Oct 06, 2014 5:48:42 PM
org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
executeTaskLocally
INFO: Executing task: [Crawler Task] locally
java.lang.NullPointerException
at
org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance.run(ExternScriptTaskInstance.java:72)
at
org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.executeTaskLocally(IterativeWorkflowProcessorThread.java:574)
at
org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread.run(IterativeWorkflowProcessorThread.java:321)
at EDU.oswego.cs.dl.util.concurrent.PooledExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Thread.java:745)
Oct 06, 2014 5:48:42 PM
org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread
executeTaskLocally
WARNING: Exception executing task: [Crawler Task] locally: Message: null
<cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas";>
<!--
TODO: Add some examples
-->
<task id="urn:oodt:crawlerTask" name="Crawler Task"
class="org.apache.oodt.cas.workflow.examples.ExternScriptTaskInstance"/>
<conditions/> <!-- There are no pre execution conditions right now -->
<configuration>
<property name="ShellType" value="/bin/sh" />
<property name="PathToScript"
value="[OODT_HOME]/crawler/bin/crawler_launcher"/>
</configuration>
<metadata>
<args>
<arg>--operation</arg>
<arg>--launchAutoCrawler</arg>
<arg>--productPath</arg>
<arg>[OODT_HOME]/data/staging</arg>
<arg>--filemgrUrl</arg>
<arg>http://localhost:9000</arg>
<arg>--clientTransferer</arg>
<arg>org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory</arg>
<arg>--mimeExtractorRepo</arg>
<arg>[$OODT_HOME]/extensions/policy/mime-extractor-map.xml</arg>
<arg>--actionIds</arg>
<arg>MoveFileToLevel0Dir</arg>
</args>
</metadata>
</cas:tasks>
Valerie A. Mallder
New Horizons Deputy Mission System Engineer
The Johns Hopkins University/Applied Physics Laboratory
11100 Johns Hopkins Rd (MS 23-282), Laurel, MD 20723
240-228-7846 (Office) 410-504-2233 (Blackberry)
