If these latencies will stay constant, Airavata will show Failure of some jobs which run over than 60 seconds. However, My use cases have many kind of jobs which run very long time(more than a few hours) Can I use Airavata for my use cases?
Shinyoung From: Suresh Marru [mailto:[email protected]] Sent: Wednesday, April 10, 2013 10:55 AM To: [email protected] Subject: Re: Another 3 question on computation time and managing experiment recoreds. Hi Shinyoung, On Apr 4, 2013, at 1:32 AM, 안신영 <[email protected]<mailto:[email protected]>> wrote: Suresh, thanks to your comment! I solved the problem linking two applications. Now, I have other questions!! 1) Why Computation time for each application takes about 60 seconds even though the application finish earlier than 60 seconds? In his case, I think, the application finished earlier, so computation time should be shorter than 60 sec. As with any abstractions, you will have latencies, as illustrated in [1], there are 4 layers from job, hence the delay. In an ideal world, everything will be push bases, instead of polling so if these latencies are too much of a bottle neck, I encourage you to investigate further and report as a feature request where you would like to see it optimized. 2) What will happening if application run more than 60 seconds? Generally these latencies will stay constant. 3) How do I delete Experiments records? I do not think delete Experiments is enabled in the GUI. Can you please create a JIRA at [2]. Suresh [1] - https://cwiki.apache.org/confluence/display/AIRAVATA/Globus+Grid+Integration [2] - https://issues.apache.org/jira/browse/airavata Thanks in advance! Shinyoung From: Suresh Marru [mailto:[email protected]<http://apache.org>] Sent: Thursday, April 04, 2013 12:40 PM To: [email protected]<mailto:[email protected]> Subject: Re: How can I use File Type when I register web services? Hi Shinyoung, I understand your confusion. Lets understand how Airavata determines outputs. Since Airavata Generic Application Factory manages applications of various types, it needs some standard way of determine outputs. The simplest approach is to have all of standard out passed to next application. But that does not work well, because the stdout can be very large, and secondly as you have seen with your applications, most of the scientific applications write to files. So for any basic primitive types, Airavata looks for name=value in stdout. So for your bwa_aln application output, you will need to change the output type to URI. Secondly, you bwa_aln should printout the output variable name=path_to_file. So something like saiFile=/work/test_temp_created_dir/file1.output. If you cannot modify the source code of bwa_aln, the easiest solution is to wrap it with a shell script and at the end of the shell script you can echo these values. An alternative mechanism is to write any files you need to the outputData sub directory. As you can see the working directory created it will have two sub directories, inputData and outputData. If name=value is not found in stdout, then Airavata will pick any files within the outputData and assume they are the required files. If you use String as the output type, then the value /work/test_temp_created_dir/file1.output will be passed to second application. If URL is selected then it will be passed as a file. The later one is very useful if your second application is running on a different machine. In which case Airavata will first move the file and then execute the second application. Hope this helps, Suresh On Apr 3, 2013, at 10:36 PM, 안신영 <[email protected]<mailto:[email protected]>> wrote: Thanks Suresh! But, I’m not clear about using arguments. As I understand, File Type should be used for file staging from the node of airavata server to node of globus server. Am I correct? If so, I want to talk about my use case : First application(bwa_aln) generate file, second application(bwa_sampe) should start after execution of first application and should receive the file name, which will be generated by first application. The file name which first application generate is input argument of first application. As I understand, for the sequential execution of these two applications, two application should be linked with each other so that second application get the output argument of first application. Next figure shows my workflow. I want to use String type as first application’s output type and use String type as second application’s input type. The String type argument means filename(saiFile below). <image001.png> such workflow returns errors as follows [ERROR] Data for the output parameter 'saiFile' was not found org.apache.airavata.core.gfac.exception.ProviderException: Data for the output parameter 'saiFile' was not found at org.apache.airavata.core.gfac.provider.impl.GramProvider.processOutput(GramProvider.java:369) at org.apache.airavata.core.gfac.provider.AbstractProvider.execute(AbstractProvider.java:76) at org.apache.airavata.core.gfac.services.impl.AbstractSimpleService.execute(AbstractSimpleService.java:118) at org.apache.airavata.core.gfac.GfacAPI.gridJobSubmit(GfacAPI.java:140) at org.apache.airavata.xbaya.invoker.EmbeddedGFacInvoker.invoke(EmbeddedGFacInvoker.java:256) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.handleWSComponent(WorkflowInterpreter.java:749) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.executeDynamically(WorkflowInterpreter.java:533) at org.apache.airavata.xbaya.interpretor.WorkflowInterpreter.scheduleDynamically(WorkflowInterpreter.java:218) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.executeWorkflow(WorkflowInterpretorSkeleton.java:389) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton.access$400(WorkflowInterpretorSkeleton.java:87) at org.apache.airavata.xbaya.interpretor.WorkflowInterpretorSkeleton$2.run(WorkflowInterpretorSkeleton.java:382) at java.lang.Thread.run(Thread.java:636) when the output type of first application is String, Why Airavata throw errors? How do I have to handle these error? Thanks in advance. Shinyoung. From: Suresh Marru [mailto:[email protected]<http://apache.org>] Sent: Wednesday, April 03, 2013 9:44 PM To: [email protected]<mailto:[email protected]> Subject: Re: How can I use File Type when I register web services? Hi Shinyoug, Please read this thread if it will answer your questions - http://markmail.org/thread/hd7azhp7w7o7eqyq Meanwhile I will work on a wiki page to address specifically your questions. These are good FAQ's. We will appreciate if you can contribute from your learning as well. Cheers, Suresh On Apr 3, 2013, at 5:00 AM, 안신영 <[email protected]<mailto:[email protected]>> wrote: When I select File type on application parameter, I wonder how File type is used. I have some applications which read files and generate new file. And second application must use the file generated by first application. In this case, may I use File type as first application’s output type and second application’s input type? If not, do I have to use String type to pass the name of file generated by first application? I need a more detailed tutorials, workflow samples.. Regards, Shinyoung
