Good advice, Val. I think I may know the issue here, though. I believe the issue is *definitely* associated with the legacy keys issue, as you stated. If you roll down to the NPE that Andrea is experience, it’s because one of the PGE config key values is null, one of the required ones that OODT expects. Pro tip: filing a JIRA ticket for actually stating which one of these is null would be IMMENSELY helpful and a relatively simple fix. Andrea, or Val, if one of you could file that JIRA ticket I will commit to updating PGE to print out which key is actually null. This one has been a long time coming.
Now: why is that key null? It’s null b/c I’m almost positive that the legancyMode for PGE is turned on by default in RADIX. Andrea: what happens if you grab this file: http://s.apache.org/kDs and replace your $OODT_HOME/resmgr/bin/batch_stub file with it. Stop the batch stub, turn it back on, submit the job again, and cross your fingers! Cheers, Chris ---- Chris Mattmann chris.mattm...@gmail.com -----Original Message----- From: "Mallder, Valerie" <valerie.mall...@jhuapl.edu> Reply-To: <dev@oodt.apache.org> Date: Tuesday, June 9, 2015 at 8:05 AM To: "dev@oodt.apache.org" <dev@oodt.apache.org> Subject: RE: OODT Documentation >Hi Andrea, > >I have found it helpful when debugging, or just investigating how >something works, to remove the resource manager from the equation and run >without it at first. If you can get your tasks to run without the >resource manager, then you should be able to get them to run WITH the >resource manager. And if you can't, then at least you've narrowed down >the area where the problem is located. > >Go into oodt/workflow/etc and edit the file workflow.properties and >comment out line 36. (Line 39 if you are using the trunk version) This >should be the line that specifies the resource manager url. Just comment >that out. And then try running again. You will of course get a >different set of log messages, but they might not look so cryptic this >time. > >Let me know how it goes. >Val > > > > >Valerie A. Mallder >New Horizons Deputy Mission System Engineer >Johns Hopkins University/Applied Physics Laboratory > > >> -----Original Message----- >> From: Andrea DeMarco [mailto:dr.andrea.dema...@gmail.com] >> Sent: Tuesday, June 09, 2015 10:37 AM >> To: Mattmann, Chris A (3980) >> Cc: Wyngaard, Jane R (3980-Affiliate); Starch, Michael D (398M); >>Kristian ZARB >> ADAMI; Ramirez, Paul M (398M); Alessio Magro; dev@oodt.apache.org >> Subject: Re: OODT Documentation >> >> Hi Chris, >> >> Some additional detail: >> >> When stopping/starting the workflow manager before kicking the PGE, the >>terminal >> (post PGE submission) also prints out the following: >> >> andrea@issa:~/oodt/workflow/bin$ ./wmgr stop >>andrea@issa:~/oodt/workflow/bin$ >> ./wmgr start Workflow Manager started PID file >> (/home/andrea/oodt/workflow/run/cas.workflow.pid). >> andrea@issa:~/oodt/workflow/bin$ Jun 09, 2015 4:34:24 PM >> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager loadProperties >> INFO: Loading Workflow Manager Configuration Properties from: >> [/home/andrea/oodt/workflow/etc/workflow.properties] >> Jun 09, 2015 4:34:25 PM >> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager <init> >> INFO: Workflow Manager started by andrea Jun 09, 2015 4:34:29 PM >> org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager handleEvent >> INFO: WorkflowManager: Received event: fileconcatenator-pge Jun 09, 2015 >> 4:34:29 PM org.apache.oodt.cas.workflow.system.XmlRpcWorkflowManager >> handleEvent >> INFO: WorkflowManager: Workflow FileConcatenatorWorkflow retrieved for >>event >> fileconcatenator-pge Jun 09, 2015 4:34:29 PM >> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread >> setRClient >> INFO: Resource Manager Job Submission enabled to: >>[http://localhost:9002] Jun >> 09, 2015 4:34:29 PM >> org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread >> checkTaskRequiredMetadata >> INFO: All required metadata fields present for task: [FileConcatenator] >>Jun 09, >> 2015 4:34:29 PM >> org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient >>submitJob >> FINEST: [{job.queueName=high, >> job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob, >> job.name=urn:oodt:FileConcatenator, >> job.id=, job.status=, job.load=2, >> job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput}, >> >>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.FileCo >>ncaten >> atorPGETask, >> >>task.config={PGETask_ConfigFilePath=/home/andrea/oodt/pge/file_concatenat >>or/ >> pge-configs/PGEConfig.xml, >> >>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransfer >>.Local >> DataTransferFactory, >> >>PCS_ActionRepoFile=file:/home/andrea/oodt/crawler/policy/crawler-config.x >>ml, >> PCS_MetFileExtension=met, PGETask_DumpMetadata=true, >> PCS_WorkflowManagerUrl= http://localhost:9001, >> PCS_FileManagerUrl=http://localhost:9000, >> PGETask_Name=FileConcatenator}, >> task.metadata={TaskId=[urn:oodt:FileConcatenator], WorkflowManagerUrl=[ >> http://issa:9001], JobId=[a2eecfc3-0eb4-11e5-ac1b-fff04e088d71], >> ProcessingNode=[issa], RunID=[testNumber1], >>WorkflowInstId=[a2eecfc3-0eb4- >> 11e5-ac1b-fff04e088d71]}}] >> >> Not sure what it all means, but it may help :) >> >> Cheers, >> Andrea >> >> On 9 June 2015 at 16:03, Andrea DeMarco <dr.andrea.dema...@gmail.com> >> wrote: >> >> > Hi Chris, >> > >> > No problem - I can imagine that checklist to be quite extensive :) >> > >> > I'll try and help as best I can: >> > >> > 1) The output when starting the batch_stub is: >> > andrea@issa:~/oodt/resmgr/bin$ ./batch_stub 2001 [1] 2535 >> > andrea@issa:~/oodt/resmgr/bin$ Jun 09, 2015 3:53:04 PM >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub <init> >> > INFO: XmlRpc Batch Stub started by andrea >> > >> > 2) The latest oodt.out log file (after running the below) is attached >>. >> > >> > 3) Kicking of the PGE with the following command (and terminal >>output): >> > >> > andrea@issa:~/oodt/workflow/bin$ ./wmgr-client --url >> > http://localhost:9001 --operation --sendEvent --eventName >> > fileconcatenator-pge --metaData --key RunID testNumber1 Sending event >> > 'fileconcatenator-pge'... SUCCESS >> > >> > I have now shut down my VM and restarted everything. When kicking off >> > the PGE, this is the terminal output from the same terminal that >> > started the >> > batch_stub: >> > >> > Jun 09, 2015 3:57:26 PM >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub >> > genericExecuteJob >> > INFO: stub attempting to execute class: >> > [org.apache.oodt.cas.workflow.structs.TaskJob] >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance >> > createPgeMetadata >> > INFO: Converting workflow configuration to static metadata... >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance >> > createPgeMetadata >> > INFO: Loading workflow context metadata... >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance >> > createPgeConfig >> > INFO: Create PgeConfig... >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance >> > createPgeConfig >> > INFO: Using default PgeConfigBuilder: >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder >> > Jun 09, 2015 3:57:26 PM org.apache.oodt.cas.pge.PGETaskInstance run >> > SEVERE: PGETask FAILED!!! : Failed to build PgeConfig : Failed to >> > parse >> > value: null >> > java.io.IOException: Failed to build PgeConfig : Failed to parse >>value: >> > null >> > at >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg >> > eConfigBuilder.java:87) >> > at >> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc >> > e.java:231) at >> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) >> > at >> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) >> > at >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob >> > .run(XmlRpcBatchStub.java:229) at >> > java.lang.Thread.run(Thread.java:745) >> > Caused by: java.lang.Exception: Failed to parse value: null at >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) >> > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) >> > at >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg >> > eConfigBuilder.java:77) >> > ... 5 more >> > Caused by: java.lang.NullPointerException at >> > java.util.regex.Matcher.getTextLength(Matcher.java:1234) >> > at java.util.regex.Matcher.reset(Matcher.java:308) >> > at java.util.regex.Matcher.<init>(Matcher.java:228) >> > at java.util.regex.Pattern.matcher(Pattern.java:1088) >> > at >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla >> > cement(PathUtils.java:321) >> > at >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU >> > tils.java:96) at >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486) >> > ... 7 more >> > >> > org.apache.oodt.cas.resource.structs.exceptions.JobInputException: >> > Failed to run task at >> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:104) >> > at >> > org.apache.oodt.cas.resource.system.extern.XmlRpcBatchStub$RunnableJob >> > .run(XmlRpcBatchStub.java:229) at >> > java.lang.Thread.run(Thread.java:745) >> > Caused by: >> > >> >>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExcep >>tion >> : >> > PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse value: >> > null at >> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150) >> > at >> > org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) >> > ... 2 more >> > Caused by: java.io.IOException: Failed to build PgeConfig : Failed to >> > parse value: null at >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg >> > eConfigBuilder.java:87) >> > at >> > org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstanc >> > e.java:231) at >> > org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) >> > ... 3 more >> > Caused by: java.lang.Exception: Failed to parse value: null at >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) >> > at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) >> > at >> > org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePg >> > eConfigBuilder.java:77) >> > ... 5 more >> > Caused by: java.lang.NullPointerException at >> > java.util.regex.Matcher.getTextLength(Matcher.java:1234) >> > at java.util.regex.Matcher.reset(Matcher.java:308) >> > at java.util.regex.Matcher.<init>(Matcher.java:228) >> > at java.util.regex.Pattern.matcher(Pattern.java:1088) >> > at >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisRepla >> > cement(PathUtils.java:321) >> > at >> > org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathU >> > tils.java:96) at >> > org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:486) >> > ... 7 more >> > >> > >> > >> > Cheers, >> > Andrea >> > >> > >> > On 9 June 2015 at 15:47, Mattmann, Chris A (3980) < >> > chris.a.mattm...@jpl.nasa.gov> wrote: >> > >> >> Thanks Andrea, OK bear with me as we go through our checklist :) >> >> >> >> Does anything show in $OODT_HOME/log/oodt.log? >> >> >> >> How are you kicking off the PGE? Does the terminal in which you >> >> started the batch stub print anything? >> >> >> >> Cheers, >> >> Chris >> >> >> >> >> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> +++++ >> >> Chris Mattmann, Ph.D. >> >> Chief Architect >> >> Instrument Software and Science Data Systems Section (398) NASA Jet >> >> Propulsion Laboratory Pasadena, CA 91109 USA >> >> Office: 168-519, Mailstop: 168-527 >> >> Email: chris.a.mattm...@nasa.gov >> >> WWW: http://sunset.usc.edu/~mattmann/ >> >> >> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> +++++ >> >> Adjunct Associate Professor, Computer Science Department University >> >> of Southern California, Los Angeles, CA 90089 USA >> >> >> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> +++++ >> >> >> >> >> >> >> >> >> >> -----Original Message----- >> >> From: Andrea DeMarco <dr.andrea.dema...@gmail.com> >> >> Date: Tuesday, June 9, 2015 at 6:44 AM >> >> To: jpluser <chris.a.mattm...@jpl.nasa.gov> >> >> Cc: "Wyngaard, Jane R (3980-Affiliate)" >> >> <jane.r.wynga...@jpl.nasa.gov>, "Starch, Michael D (398M)" >> >> <michael.d.sta...@jpl.nasa.gov>, Kristian ZARB ADAMI >> <k...@astro.ox.ac.uk>, "Ramirez, Paul M (398M)" >> >> <paul.m.rami...@jpl.nasa.gov>, Alessio Magro <les...@gmail.com>, >> >> "dev@oodt.apache.org" <dev@oodt.apache.org> >> >> Subject: Re: OODT Documentation >> >> >> >> >Hi Chris, >> >> > >> >> > >> >> >Yup, the jar file is present in both resmgr/lib and workflow/lib. >> >> >You can have a look at my JAR contents at: >> >> > >> >> > >> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja >> >> r?dl= >> >> >0 >> >> >< >> >> https://www.dropbox.com/s/1h0siibxf8sv5ut/fileconcatenator-pge-1.0.ja >> >> r?dl >> >> >=0> - perhaps something's off there? >> >> > >> >> > >> >> >Cheers, >> >> >Andrea >> >> > >> >> >> >> >> > >> > >> >> >> -- >> Best Regards, >> Andrea DeMarco >> Institute of Space Sciences and Astronomy University of Malta
