Hi Andrea, I have found it helpful when debugging, or just investigating how something works, to remove the resource manager from the equation and run without it at first. If you can get your tasks to run without the resource manager, then you should be able to get them to run WITH the resource manager. And if you can't, then at least you've narrowed down the area where the problem is located.
Go into oodt/workflow/etc and edit the file workflow.properties and comment out line 36. (Line 39 if you are using the trunk version) This should be the line that specifies the resource manager url. Just comment that out. And then try running again. You will of course get a different set of log messages, but they might not look so cryptic this time. Let me know how it goes. Val Valerie A. Mallder New Horizons Deputy Mission System Engineer Johns Hopkins University/Applied Physics Laboratory > -----Original Message----- > From: Andrea DeMarco [mailto:dr.andrea.dema...@gmail.com] > Sent: Tuesday, June 09, 2015 10:37 AM > To: Mattmann, Chris A (3980) > Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M); Kristian ZARB > ADAMI; Ramirez, Paul M (398M); Alessio Magro; dev@oodt.apache.org > Subject: Re: OODT Documentation > > Hi Chris, > > Some additional detail: > > When stopping/starting the workflow manager before kicking the PGE, the > terminal > (post PGE submission) also prints out the following: > > andrea@issa:~/oodt/workflow/bin$ ./wmgr stop andrea@issa:~/oodt/workflow/bin$ > ./wmgr start Workflow Manager started PID file > (/home/andrea/oodt/workflow/run/cas.workflow.pid). > andrea@issa:~/oodt/workflow/bin$ Jun 09, 2015 4:34:24 PM > org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties > INFO: Loading Workflow Manager Configuration Properties from: > [/home/andrea/oodt/workflow/etc/workflow.properties] > Jun 09, 2015 4:34:25 PM > org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init> > INFO: Workflow Manager started by andrea Jun 09, 2015 4:34:29 PM > org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent > INFO: WorkflowManager: Received event: fileconcatenator-pge Jun 09, 2015 > 4:34:29 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager > handleEvent > INFO: WorkflowManager: Workflow FileConcatenatorWorkflow retrieved for event > fileconcatenator-pge Jun 09, 2015 4:34:29 PM > org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread > setRClient > INFO: Resource Manager Job Submission enabled to: [http://localhost:9002] Jun > 09, 2015 4:34:29 PM > org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread > checkTaskRequiredMetadata > INFO: All required metadata fields present for task: [FileConcatenator] Jun > 09, > 2015 4:34:29 PM > org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient submitJob > FINEST: [{job.queueName=high, > job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob, > job.name=urn:oodt:FileConcatenator, > job.id=, job.status=, job.load=2, > job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput}, > {task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileConcaten > atorPGETask, > task.config={PGETask_ConfigFilePath=/home/andrea/oodt/pge/file_concatenator/ > pge-configs/PGEConfig.xml, > PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer.Local > DataTransferFactory, > PCS_ActionRepoFile=file:/home/andrea/oodt/crawler/policy/crawler-config.xml, > PCS_MetFileExtension=met, PGETask_DumpMetadata=true, > PCS_WorkflowManagerUrl= http://localhost:9001, > PCS_FileManagerUrl=http://localhost:9000, > PGETask_Name=FileConcatenator}, > task.metadata={TaskId=[urn:oodt:FileConcatenator], WorkflowManagerUrl=[ > http://issa:9001], JobId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71], > ProcessingNode=[issa], RunID=[testNumber1], WorkflowInstId=[a2eecfc3-0eb4- > 11e5-ac1b-fff04e088d71]}}] > > Not sure what it all means, but it may help :) > > Cheers, > Andrea > > On 9 June 2015 at 16:03, Andrea DeMarco <dr.andrea.dema...@gmail.com> > wrote: > > > Hi Chris, > > > > No problem - I can imagine that checklist to be quite extensive :) > > > > I'll try and help as best I can: > > > > 1) The output when starting the batch_stub is: > > andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001 [1] 2535 > > andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM > > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init> > > INFO: XmlRpc Batch Stub started by andrea > > > > 2) The latest oodt.out log file (after running the below) is attached . > > > > 3) Kicking of the PGE with the following command (and terminal output): > > > > andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url > > http://localhost:9001 --operation --sendEvent --eventName > > fileconcatenator-pge --metaData --key RunID testNumber1 Sending event > > 'fileconcatenator-pge'... SUCCESS > > > > I have now shut down my VM and restarted everything. When kicking off > > the PGE, this is the terminal output from the same terminal that > > started the > > batch_stub: > > > > Jun 09, 2015 3:57:26 PM > > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub > > genericExecuteJob > > INFO: stub attempting to execute class: > > [org.apache.oodt.cas.workflow.structs.TaskJob] > > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance > > createPgeMetadata > > INFO: Converting workflow configuration to static metadata... > > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance > > createPgeMetadata > > INFO: Loading workflow context metadata... > > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance > > createPgeConfig > > INFO: Create PgeConfig... > > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance > > createPgeConfig > > INFO: Using default PgeConfigBuilder: > > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder > > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run > > SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to > > parse > > value: null > > java.io.IOException: Failed to build PgeConfig : Failed to parse value: > > null > > at > > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg > > eConfigBuilder.java:87) > > at > > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc > > e.java:231) at > > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) > > at > > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) > > at > > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob > > .run(XmlRpcBatchStub.java:229) at > > java.lang.Thread.run(Thread.java:745) > > Caused by: java.lang.Exception: Failed to parse value: null at > > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) > > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) > > at > > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg > > eConfigBuilder.java:77) > > ... 5 more > > Caused by: java.lang.NullPointerException at > > java.util.regex.Matcher.getTextLength(Matcher.java:1234) > > at java.util.regex.Matcher.reset(Matcher.java:308) > > at java.util.regex.Matcher.<init>(Matcher.java:228) > > at java.util.regex.Pattern.matcher(Pattern.java:1088) > > at > > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla > > cement(PathUtils.java:321) > > at > > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU > > tils.java:96) at > > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486) > > ... 7 more > > > > org.apache.oodt.cas.resource.structs.exceptions.JobInputException: > > Failed to run task at > > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104) > > at > > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob > > .run(XmlRpcBatchStub.java:229) at > > java.lang.Thread.run(Thread.java:745) > > Caused by: > > > org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceException > : > > PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value: > > null at > > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150) > > at > > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) > > ... 2 more > > Caused by: java.io.IOException: Failed to build PgeConfig : Failed to > > parse value: null at > > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg > > eConfigBuilder.java:87) > > at > > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc > > e.java:231) at > > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) > > ... 3 more > > Caused by: java.lang.Exception: Failed to parse value: null at > > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) > > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) > > at > > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg > > eConfigBuilder.java:77) > > ... 5 more > > Caused by: java.lang.NullPointerException at > > java.util.regex.Matcher.getTextLength(Matcher.java:1234) > > at java.util.regex.Matcher.reset(Matcher.java:308) > > at java.util.regex.Matcher.<init>(Matcher.java:228) > > at java.util.regex.Pattern.matcher(Pattern.java:1088) > > at > > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla > > cement(PathUtils.java:321) > > at > > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU > > tils.java:96) at > > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486) > > ... 7 more > > > > > > > > Cheers, > > Andrea > > > > > > On 9 June 2015 at 15:47, Mattmann, Chris A (3980) < > > chris.a.mattm...@jpl.nasa.gov> wrote: > > > >> Thanks Andrea, OK bear with me as we go through our checklist :) > >> > >> Does anything show in $OODT_HOME/log/oodt.log? > >> > >> How are you kicking off the PGE? Does the terminal in which you > >> started the batch stub print anything? > >> > >> Cheers, > >> Chris > >> > >> > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > >> Chris Mattmann, Ph.D. > >> Chief Architect > >> Instrument Software and Science Data Systems Section (398) NASA Jet > >> Propulsion Laboratory Pasadena, CA 91109 USA > >> Office: 168-519, Mailstop: 168-527 > >> Email: chris.a.mattm...@nasa.gov > >> WWW: http://sunset.usc.edu/~mattmann/ > >> > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > >> Adjunct Associate Professor, Computer Science Department University > >> of Southern California, Los Angeles, CA 90089 USA > >> > +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > +++++ > >> > >> > >> > >> > >> -----Original Message----- > >> From: Andrea DeMarco <dr.andrea.dema...@gmail.com> > >> Date: Tuesday, June 9, 2015 at 6:44 AM > >> To: jpluser <chris.a.mattm...@jpl.nasa.gov> > >> Cc: "Wyngaard, Jane R (3980-Affiliate)" > >> <jane.r.wynga...@jpl.nasa.gov>, "Starch, Michael D (398M)" > >> <michael.d.sta...@jpl.nasa.gov>, Kristian ZARB ADAMI > <k...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)" > >> <paul.m.rami...@jpl.nasa.gov>, Alessio Magro <les...@gmail.com>, > >> "dev@oodt.apache.org" <dev@oodt.apache.org> > >> Subject: Re: OODT Documentation > >> > >> >Hi Chris, > >> > > >> > > >> >Yup, the jar file is present in both resmgr/lib and workflow/lib. > >> >You can have a look at my JAR contents at: > >> > > >> > > >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja > >> r?dl= > >> >0 > >> >< > >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja > >> r?dl > >> >=0> - perhaps something's off there? > >> > > >> > > >> >Cheers, > >> >Andrea > >> > > >> > >> > > > > > > > -- > Best Regards, > Andrea DeMarco > Institute of Space Sciences and Astronomy University of Malta
