Hi Raminder, I tested the task submission again. My workflow has 2 application services and 4 inputs total. Those 2 tasks run in less than 5 minutes, it is a just a test. The first application services ran successfully (File transfer and job), but the second application services didn't. The second application services got the files transfer correctly, but did not submit the task to PBS. The Airavata server was constantly printing "Job Error Code: 72". Do you know what could possibly cause this message?
This is the Airavata-Server log messages: [INFO] -----END DATA----- [INFO] Status is zero [INFO] Status of job https://gridftp1.ls4.tacc.utexas.edu:50383/16289984330111623786/8943296923859958664/isFAILED [INFO] -----DATA----- [INFO] Status of job https://gridftp1.ls4.tacc.utexas.edu:50383/16289984330111623786/8943296923859958664/isFAILED [INFO] -----END DATA----- [INFO] Job Error Code: 72 [ERROR] Context passed was NULL. java.lang.RuntimeException: Context passed was NULL. at org.apache.airavata.workflow.tracking.impl.ProvenanceNotifierImpl.sendingFault(ProvenanceNotifierImpl.java:496) at org.apache.airavata.workflow.tracking.impl.ProvenanceNotifierImpl.sendingFault(ProvenanceNotifierImpl.java:485) at org.apache.airavata.core.gfac.notification.impl.WorkflowTrackingNotification.executionFail(WorkflowTrackingNotification.java:108) at org.apache.airavata.core.gfac.notification.impl.DefaultNotifier.executionFail(DefaultNotifier.java:135) at org.apache.airavata.core.gfac.provider.impl.GramProvider.executeApplication(GramProvider.java:225) at org.apache.airavata.core.gfac.provider.AbstractProvider.execute(AbstractProvider.java:69) at org.apache.airavata.core.gfac.services.impl.AbstractSimpleService.execute(AbstractSimpleService.java:118) at org.apache.airavata.core.gfac.GfacAPI.gridJobSubmit(GfacAPI.java:140) at org.apache.airavata.xbaya.invoker.EmbeddedGFacInvoker.invoke(EmbeddedGFacInvoker.java:256) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.handleWSComponent(WorkflowInterpreter.java:749) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.executeDynamically(WorkflowInterpreter.java:533) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.scheduleDynamically(WorkflowInterpreter.java:218) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.executeWorkflow(WorkflowInterpretorSkeleton.java:389) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.access$400(WorkflowInterpretorSkeleton.java:87) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton$2.run(WorkflowInterpretorSkeleton.java:382) at java.lang.Thread.run(Thread.java:680) [INFO] -----DATA----- [INFO] Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 [INFO] -----END DATA----- [ERROR] Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 org.apache.airavata.core.gfac.exception.JobSubmissionFault: Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 at org.apache.airavata.core.gfac.provider.impl.GramProvider.executeApplication(GramProvider.java:222) at org.apache.airavata.core.gfac.provider.AbstractProvider.execute(AbstractProvider.java:69) at org.apache.airavata.core.gfac.services.impl.AbstractSimpleService.execute(AbstractSimpleService.java:118) at org.apache.airavata.core.gfac.GfacAPI.gridJobSubmit(GfacAPI.java:140) at org.apache.airavata.xbaya.invoker.EmbeddedGFacInvoker.invoke(EmbeddedGFacInvoker.java:256) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.handleWSComponent(WorkflowInterpreter.java:749) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.executeDynamically(WorkflowInterpreter.java:533) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.scheduleDynamically(WorkflowInterpreter.java:218) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.executeWorkflow(WorkflowInterpretorSkeleton.java:389) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.access$400(WorkflowInterpretorSkeleton.java:87) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton$2.run(WorkflowInterpretorSkeleton.java:382) at java.lang.Thread.run(Thread.java:680) Caused by: java.lang.Exception: Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 ... 12 more Exception in thread "Thread-67" org.apache.airavata.workflow.model.exceptions.WorkflowRuntimeException: org.apache.airavata.workflow.model.exceptions.WorkflowException: Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.executeWorkflow(WorkflowInterpretorSkeleton.java:392) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.access$400(WorkflowInterpretorSkeleton.java:87) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton$2.run(WorkflowInterpretorSkeleton.java:382) at java.lang.Thread.run(Thread.java:680) Caused by: org.apache.airavata.workflow.model.exceptions.WorkflowException: Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 at org.apache.airavata.xbaya.invoker.EmbeddedGFacInvoker.invoke(EmbeddedGFacInvoker.java:321) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.handleWSComponent(WorkflowInterpreter.java:749) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.executeDynamically(WorkflowInterpreter.java:533) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.scheduleDynamically(WorkflowInterpreter.java:218) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.executeWorkflow(WorkflowInterpretorSkeleton.java:389) ... 3 more Caused by: org.apache.airavata.core.gfac.exception.JobSubmissionFault: Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 at org.apache.airavata.core.gfac.provider.impl.GramProvider.executeApplication(GramProvider.java:222) at org.apache.airavata.core.gfac.provider.AbstractProvider.execute(AbstractProvider.java:69) at org.apache.airavata.core.gfac.services.impl.AbstractSimpleService.execute(AbstractSimpleService.java:118) at org.apache.airavata.core.gfac.GfacAPI.gridJobSubmit(GfacAPI.java:140) at org.apache.airavata.xbaya.invoker.EmbeddedGFacInvoker.invoke(EmbeddedGFacInvoker.java:256) ... 7 more Caused by: java.lang.Exception: Job Protocol : https Host name : gridftp1.ls4.tacc.utexas.edu Port number : 50383 Url path : 16289984330111623786/8943296923859958664/ User : null Pwd : null on host lonestar4.tacc.teragrid.org Job Exit Code = 72 ... 12 more Thank you, On Wed, Feb 6, 2013 at 9:23 AM, Raminder Singh <[email protected]>wrote: > Hi Pedro, > > Can you check space in home directory of your account on Lonestar? I have > seen such problem if you cross disk quota. Gram does not give any error > and job does not go into queue. If quota is fine then we need to debug more. > > Thanks > Raminder > > On Feb 5, 2013, at 7:49 PM, Pedro da Silveira wrote: > > > Hi Dev, > > > > I am trying to submit a workflow using my Xsede account using Xbaya. It > has > > worked successfully using "ogce" account. > > I changed the file "airavata-server.properties" to use my Xsede portal > > account. > > > > myproxy.user=pedrorcs > > myproxy.pass=****** > > > > I also changed the Application Service to use a different settings like > my > > user $SCRATCH directory. > > > > Executable path: > > /scratch/00091/tg458470/executePwscf.sh > > > > Scratch Working directory: > > /scratch/00091/tg458470/Phonon > > > > I set the workflow to run then I setup correctly the local path to input > > files on my desktop. > > All input files got transferred correctly, but the job were never not > > submitted to PBS. > > Can someone please clarify if I am doing something wrong? > > > > This is the log on Airavata-Server: > > > > > ================================================================================================================= > > [INFO] Experiment launched > > :SimplePhonon_01a4d374-486f-4948-937f-a9de4b2b45eb > > [INFO] -----DATA----- > > [INFO] Start scheduling > > [INFO] -----END DATA----- > > [INFO] Searching registry for some deployed application hosts > > [INFO] Found service on: lonestar4.tacc.teragrid.org > > [INFO] Found service on: lonestar4.tacc.teragrid.org > > [INFO] -----DATA----- > > [INFO] Finish scheduling > > [INFO] -----END DATA----- > > null > > [INFO] Proxy file renewed to > > /tmp/x509up_upedrorcsed4c4290-90e6-40a6-bfe7-3c5239da1b7c for the user > > pedrorcs with 3600 lifetime. > > [INFO] Creating Directory = gridftp1.ls4.tacc.utexas.edu:2811 > > =//scratch/00091/tg458470/Phonon > > [INFO] Creating Directory = gridftp1.ls4.tacc.utexas.edu:2811 > > > =//scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d > > [INFO] Creating Directory = gridftp1.ls4.tacc.utexas.edu:2811 > > > =//scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData > > [INFO] Creating Directory = gridftp1.ls4.tacc.utexas.edu:2811 > > > =//scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/outputData > > org.globus.gsi.gssapi.GlobusGSSCredentialImpl@2161df1f > > [INFO] The remote file is > > > ///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/Pwscf_Input > > [INFO] Uploading file > > [INFO] Upload file > > > to:///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/Pwscf_Input > > is done > > org.globus.gsi.gssapi.GlobusGSSCredentialImpl@2161df1f > > [INFO] The remote file is > > > ///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/Mg.vbc3 > > [INFO] Uploading file > > [INFO] Upload file > > > to:///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/Mg.vbc3 > > is done > > org.globus.gsi.gssapi.GlobusGSSCredentialImpl@2161df1f > > [INFO] The remote file is > > > ///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/008Ocabm3.vdb > > [INFO] Uploading file > > [INFO] Upload file > > > to:///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/008Ocabm3.vdb > > is done > > [INFO] -----DATA----- > > [INFO] Start execution > > [INFO] -----END DATA----- > > org.globus.gsi.gssapi.GlobusGSSCredentialImpl@2161df1f > > [INFO] -----DATA----- > > [INFO] Finished launching job, Host = lonestar4.tacc.teragrid.org RSL = > &( > > queue = "development" )( stdout = > > > "/scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/lonestar_application.stdout" > > )( count = "12" )( executable = "/scratch/00091/tg458470/executePwscf.sh" > > )( stderr = > > > "/scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/lonestar_application.stderr" > > )( maxwalltime = "20" )( hostCount = "1" )( minmemory = "1024" )( > project = > > "TG-TRA120030" )( jobtype = "mpi" )( environment = ( "inputData" > > > "/scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData" > > ) ( "outputData" > > > "/scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/outputData" > > ) )( proxy_timeout = "1" )( arguments = > > > "///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/Pwscf_Input" > > > "///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/Mg.vbc3" > > > "///scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d/inputData/008Ocabm3.vdb" > > )( directory = > > > "/scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d" > > )( maxmemory = "2048" ) working directory = > > > /scratch/00091/tg458470/Phonon/AppSimplePhonon_Tue_Feb_05_18_30_22_CST_2013_7a7dded8-e058-4be2-ad43-cacf77b3138d > > temp directory = /scratch/00091/tg458470/Phonon Globus GateKeeper > Endpoint > > = gridftp1.ls4.tacc.utexas.edu:2119/jobmanager-sge > > [INFO] -----END DATA----- > > > ================================================================================================================= > > > > > > Thank you so much, > > > > > > Pedro da Silveira > >
