Great. One last question. Can you show me the contents (paste.bin) of your wmgr script and your batch_stub script?
++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Chris Mattmann, Ph.D. Adjunct Associate Professor, Computer Science Department University of Southern California Los Angeles, CA 90089 USA Email: [email protected] WWW: http://sunset.usc.edu/~mattmann/ ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ -----Original Message----- From: Renu Kanakamedala <[email protected]> Date: Wednesday, November 5, 2014 at 10:20 PM To: Chris Mattmann <[email protected]> Cc: Meghamsh Utkur <[email protected]>, "[email protected]" <[email protected]>, Preethi Ramesh <[email protected]>, Gouthami Kondakindi <[email protected]> Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to parse null Error. >renu@ubuntu:~$ echo $WORKFLOW_HOME >/home/renu/oodt/workflow/ >renu@ubuntu:~$ echo $FILEMGR_HOME >/home/renu/oodt/filemgr >renu@ubuntu:~$ echo $WORKFLOW_URL >http://localhost:9001 >renu@ubuntu:~$ echo $FILEMGR_URL >http://localhost:9000 >renu@ubuntu:~$ echo $PGE_ROOT >/home/renu/oodt/pge/ > >renu@ubuntu:~$ echo $CRAWLER_HOME >/home/renu/oodt/crawler/ > > > > > > > >On Wed, Nov 5, 2014 at 8:38 PM, Christian Alan Mattmann ><[email protected]> wrote: > >Right, which is why I asked to specifically echo the value >of all environment variables so we would be able to see the >values. Can Renu please show the echo values? > >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >Chris Mattmann, Ph.D. >Adjunct Associate Professor, Computer Science Department >University of Southern California >Los Angeles, CA 90089 USA >Email: [email protected] >WWW: http://sunset.usc.edu/~mattmann/ >++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > >-----Original Message----- >From: Meghamsh Utkur <[email protected]> >Date: Wednesday, November 5, 2014 at 9:35 PM >To: Renu Kanakamedala <[email protected]> >Cc: Chris Mattmann <[email protected]>, "[email protected]" ><[email protected]>, Preethi Ramesh <[email protected]>, Gouthami >Kondakindi <[email protected]> >Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to >parse null Error. > >>Professor, >> >> >>To add to what Renu has said, I went through the source code and I >>suspect that the value here in this method is passed as null. >> >>XmlFilePgeConfigBuilder.java >> >>1. private String fillIn(String value,Metadata inputMetadata,boolean >>envReplaceRecur) >>2. throwsException { >>3. try{ >>4. while((value =PathUtils.doDynamicReplacement(value, >>inputMetadata)).contains("[")&& envReplaceRecur); >>5. if(value.toUpperCase() >>6. .matches("^\\s*SQL\\s*\\(.*\\)\\s*\\{.*\\}\\s*$")) >>7. value >>=QueryUtils.getQueryResultsAsString(newXmlRpcFileManagerClient(new >>URL(inputMetadata >>8. >>.getMetadata(PcsMetadataKeys.FILE_MANAGER_URL))).complexQuery(SqlParser.p >>a >>rseSqlQueryMethod(value))); >>9. return value; >>10. }catch(Exception e){ >>11. thrownew Exception("Failed to parse value: "+ value, e); >>12. } >>13. } >> >> >> >> >> >> >> >> >> >> >> >>On Wed, Nov 5, 2014 at 8:30 PM, Renu Kanakamedala >><[email protected]> wrote: >> >>Hello Professor, >> >> >>All the environment variables present in tasks.xml i.e. >>$WORKFLOW_URL,$PGE_ROOT, $FILEMGR_URL,$CRWALER_HOME file are defined. >> >> >>Please let us know how to proceed. >> >> >>Regards, >>Renu >> >> >>On Wed, Nov 5, 2014 at 8:15 PM, Christian Alan Mattmann >><[email protected]> wrote: >> >>Can you please confirm that all of the bracketed environment >>variables below do in fact exist in the shell that you are >>running ./oodt start in? >> >>E.g., please echo $ENV_VAR below and show their values. >> >>Cheers, >>Chris >> >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>Chris Mattmann, Ph.D. >>Adjunct Associate Professor, Computer Science Department >>University of Southern California >>Los Angeles, CA 90089 USA >>Email: [email protected] >>WWW: http://sunset.usc.edu/~mattmann/ >>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >> >> >> >> >>-----Original Message----- >>From: Renu Kanakamedala <[email protected]> >>Date: Wednesday, November 5, 2014 at 8:18 PM >>To: Chris Mattmann <[email protected]> >>Cc: "[email protected]" <[email protected]>, Preethi Ramesh >><[email protected]>, Gouthami Kondakindi <[email protected]> >>Subject: Re: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to >>parse null Error. >> >>>Please find below tasks.xml file. >>> >>> >>><cas:tasks xmlns:cas="http://oodt.jpl.nasa.gov/1.0/cas";> >>><!-- >>> TODO: Add some examples >>>--> >>><task id="urn:oodt:FileConcatenator" name="FileConcatenator" >>> >>>class="org.apache.oodt.pge.examples.fileconcatenator.FileConcatenatorPGE >>>T >>>a >>>sk"> >>> >>> <conditions/> >>> >>> <configuration> >>> <property name="PGETask_Name" value="FileConcatenator"/> >>> <property name="PGETask_ConfigFilePath" >>>value="[PGE_ROOT]/file_concatenator/pge-configs/PGEConfig.xml" >>>envReplace="true"/> >>> <property name="PGETask_DumpMetadata" value="true"/> >>> <property name="PCS_WorkflowManagerUrl" value="[WORKFLOW_URL]" >>>envReplace="true" /> >>> <property name="PCS_FileManagerUrl" value="[FILEMGR_URL]" >>>envReplace="true"/> >>> <property name="PCS_MetFileExtension" value="met"/> >>> <property name="PCS_ClientTransferServiceFactory" >>>value="org.apache.oodt.cas.filemgr.datatransfer.LocalDataTransferFactory >>>" >>>/ >>>> >>> <property name="PCS_ActionRepoFile" >>>value="file:[CRAWLER_HOME]/policy/crawler-config.xml" >>>envReplace="true"/> >>> </configuration> >>> >>> <requiredMetFields> >>> <metfield name="RunID"/> >>> </requiredMetFields> >>> >>></task> >>></cas:tasks> >>> >>> >>>On Wed, Nov 5, 2014 at 7:07 PM, Christian Alan Mattmann >>><[email protected]> wrote: >>> >>>Hi Renu, >>> >>>Can you please show a paste.bin of your tasks.xml? >>> >>>Cheers, >>>Chris >>> >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>>Chris Mattmann, Ph.D. >>>Adjunct Associate Professor, Computer Science Department >>>University of Southern California >>>Los Angeles, CA 90089 USA >>>Email: [email protected] >>>WWW: http://sunset.usc.edu/~mattmann/ >>>++++++++++++++++++++++++++++++++++++++++++++++++++++++++ >>> >>> >>> >>> >>>-----Original Message----- >>>From: Renu Kanakamedala <[email protected]> >>>Date: Wednesday, November 5, 2014 at 7:55 PM >>>To: Chris Mattmann <[email protected]> >>>Subject: CSCI 572 - Homework 2 - SEVERE: PGE Task FAILED! Failed to >>>parse >>>null Error. >>> >>>>Hello Professor, >> >>>> >>>> >>>>When i am trying to run file concatenator CAS PGE example using the >>>>following command. >>>> >>>> >>>>./wmgr-client --url http://localhost:9001 --operation --sendEvent >>>>--eventName fileconcatenator-pge --metaData --key RunID testNumber1 >>>> >>>> >>>> >>>>The message shown after executing this command is "Sending Event >>>>SUCCESS". Below are the log messages in workflow mgr, res mgr and batch >>>>stub. >>>> >>>> >>>>Batch Stub: >>>>Caused by: >>>>org.apache.oodt.cas.workflow.structs.exceptions.WorkflowTaskInstanceExc >>>>e >>>>p >>>>t >>>>ion: PGETask FAILED!!! : Failed to build PgeConfig : Failed to parse >>>>value: null >>>>at >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:150) >>>>at >>>>org.apache.oodt.cas.workflow.structs.TaskJob.execute(TaskJob.java:102) >>>>... 2 more >>>>Caused by: java.io.IOException: Failed to build PgeConfig : Failed to >>>>parse value: null >>>>at >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge >>>>C >>>>o >>>>n >>>>figBuilder.java:87) >>>>at >>>>org.apache.oodt.cas.pge.PGETaskInstance.createPgeConfig(PGETaskInstance >>>>. >>>>j >>>>a >>>>va:230) >>>>at >>>>org.apache.oodt.cas.pge.PGETaskInstance.run(PGETaskInstance.java:123) >>>>... 3 more >>>>Caused by: java.lang.Exception: Failed to parse value: null >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:501) >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:480) >>>>at >>>>org.apache.oodt.cas.pge.config.XmlFilePgeConfigBuilder.build(XmlFilePge >>>>C >>>>o >>>>n >>>>figBuilder.java:77) >>>>... 5 more >>>>Caused by: java.lang.NullPointerException >>>>at java.util.regex.Matcher.getTextLength(Matcher.java:1234) >>>>at java.util.regex.Matcher.reset(Matcher.java:308) >>>>at java.util.regex.Matcher.<init>(Matcher.java:228) >>>>at java.util.regex.Pattern.matcher(Pattern.java:1088) >>>>at >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicDateToMillisReplac >>>>e >>>>m >>>>e >>>>nt(PathUtils.java:321) >>>>at >>>>org.apache.oodt.cas.metadata.util.PathUtils.doDynamicReplacement(PathUt >>>>i >>>>l >>>>s >>>>.java:96) >>>>at org.apache.oodt.cas.pge.util.XmlHelper.fillIn(XmlHelper.java:488) >>>>... 7 more >>>> >>>> >>>> >>>> >>>> >>>>RES mngr: >>>> >>>> >>>>Nov 05, 2014 5:26:28 PM org.apache.oodt.cas.resource.jobqueue.JobStack >>>>addJob >>>>INFO: Added Job: [2014-11-05T17:26:28.785-08:00] to queue >>>>Nov 05, 2014 5:26:30 PM >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler run >>>>INFO: Obtained Job: [2014-11-05T17:26:28.785-08:00] from Queue: >>>>Scheduling for execution >>>>Nov 05, 2014 5:26:30 PM >>>>org.apache.oodt.cas.resource.scheduler.LRUScheduler schedule >>>>INFO: Assigning job: [urn:oodt:FileConcatenator] to node: [localhost] >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>plainConnect >>>>FINEST: ProxySelector Request for >>> >>> >> >> >>>>http://localhost:2001/ <http://localhost:2001/> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>plainConnect >>>>FINEST: Proxy used: DIRECT >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>writeRequests >>>>FINE: sun.net.www.MessageHeader@61fc61a79 pairs: {POST / HTTP/1.1: >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: >>>>text/html, image/gif, image/jpeg, *; q=.2, */*; >>>> q=.2}{Connection: keep-alive}{Content-Length: 105} >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>getInputStream >>>>FINE: sun.net.www.MessageHeader@90ff2a15 pairs: {null: HTTP/1.1 200 >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: >>>>text/xml}{Content-Length: 121} >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>plainConnect >>>>FINEST: ProxySelector Request for >>>>http://localhost:2001/ <http://localhost:2001/> >> >>>>Nov 05, 2014 5:26:30 PM sun.net.www.protocol.http.HttpURLConnection >>>>plainConnect >>>>FINEST: Proxy used: DIRECT >>>>Nov 05, 2014 5:26:31 PM sun.net.www.protocol.http.HttpURLConnection >>>>writeRequests >>>>FINE: sun.net.www.MessageHeader@2f33cbcc9 pairs: {POST / HTTP/1.1: >>>>null}{Content-Type: text/xml}{Cache-Control: no-cache}{Pragma: >>>>no-cache}{User-Agent: Java/1.7.0_65}{Host: localhost:2001}{Accept: >>>>text/html, image/gif, image/jpeg, *; q=.2, */*; >>>> q=.2}{Connection: keep-alive}{Content-Length: 2536} >>>>Nov 05, 2014 5:26:32 PM sun.net.www.protocol.http.HttpURLConnection >>>>getInputStream >>>>FINE: sun.net.www.MessageHeader@7188d6495 pairs: {null: HTTP/1.1 200 >>>>OK}{Server: Apache XML-RPC 1.0}{Connection: close}{Content-Type: >>>>text/xml}{Content-Length: 121} >>>>Nov 05, 2014 5:26:32 PM >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy run >>>>SEVERE: Job execution failed for jobId '2014-11-05T17:26:28.785-08:00' >>>>: >>>>batchstub.executeJob returned false >>>>java.lang.Exception: batchstub.executeJob returned false >>>>at >>>>org.apache.oodt.cas.resource.batchmgr.XmlRpcBatchMgrProxy.run(XmlRpcBat >>>>c >>>>h >>>>M >>>>grProxy.java:125) >>>> >>>> >>>> >>>> >>>> >>>>Workflow: >>>> >>>> >>>>Nov 05, 2014 5:26:27 PM >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread >>>>checkTaskRequiredMetadata >>>>INFO: All required metadata fields present for task: [FileConcatenator] >>>>Nov 05, 2014 5:26:28 PM >>>>org.apache.oodt.cas.resource.system.XmlRpcResourceManagerClient >>>>submitJob >>>>FINEST: [{job.queueName=high, >>>>job.instanceClassName=org.apache.oodt.cas.workflow.structs.TaskJob, >>> >>> >> >> >> >> >>>>job.name <http://job.name> <http://job.name> <http://job.name> >>>><http://job.name>=urn:oodt:FileConcatenator, > > >>>>job.id <http://job.id> <http://job.id> <http://job.id> >>>><http://job.id>=, job.status=, >>>>job.load=2, >>>>job.inputClassName=org.apache.oodt.cas.workflow.structs.TaskJobInput}, >>>>{task.instance.class=org.apache.oodt.pge.examples.fileconcatenator.File >>>>C >>>>o >>>>n >>>>catenatorPGETask, >>>>task.config={PGETask_ConfigFilePath=null/file_concatenator/pge-configs/ >>>>P >>>>G >>>>E >>>>Config.xml, >>>> >>>>PCS_ClientTransferServiceFactory=org.apache.oodt.cas.filemgr.datatransf >>>>e >>>>r >>>>. >>>>LocalDataTransferFactory, >>>>PCS_ActionRepoFile=file:/home/renu/oodt/crawler/policy/crawler-config.x >>>>m >>>>l >>>>, >>>> PCS_MetFileExtension=met, PGETask_DumpMetadata=true, >>>>PCS_WorkflowManagerUrl=http://localhost:9001, >>>> PCS_FileManagerUrl=http://localhost:9000, >>>>PGETask_Name=FileConcatenator}, >>>>task.metadata={TaskId=[urn:oodt:FileConcatenator], >>>>WorkflowManagerUrl=[http://ubuntu:9001], >>>>JobId=[ec8a60b4-6553-11e4-a26b-d92795c16b95], >>>> ProcessingNode=[ubuntu], RunID=[testNumber1], >>>>WorkflowInstId=[ec8a60b4-6553-11e4-a26b-d92795c16b95]}}] >>>>Nov 05, 2014 5:26:39 PM >>>>org.apache.oodt.cas.workflow.engine.IterativeWorkflowProcessorThread >>>>stop >>>>WARNING: Attempt to kill current resmgr job: >>>>[2014-11-05T17:26:28.785-08:00]: failed >>>> >>>> >>>>When i searched online i found one of your mail archives where you had >>>>mentioned to change tasks.xml >>>>https://www.mail-archive.com/[email protected]/msg04403.html >>>> >>>> >>>> >>>>It did not work. Please let us know how to proceed. >>>> >>>> >>>> >>>>Regards, >>>>Renu >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>> >>> >>> >>> >>> >>> >>> >>> >>> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >>-- > > >>Meghamsh UtkurGraduate Student,University of Southern California >>[email protected] >> >>(503)881-4791 <tel:%28503%29881-4791> >> >> >> > > > > > >
