Here is the issue[1]. [1]https://issues.apache.org/jira/browse/AIRAVATA-869
Lahiru On Fri, Jun 14, 2013 at 10:59 AM, Lahiru Gunathilake <[email protected]>wrote: > I got to know about this yesterday, sorry I couldn't create a jira yet. > Will do now ! > > Lahiru > > > On Fri, Jun 14, 2013 at 10:56 AM, Marlon Pierce <[email protected]> wrote: > >> -----BEGIN PGP SIGNED MESSAGE----- >> Hash: SHA1 >> >> OK--which Jira task? Did it get marked as a blocker for 0.8? >> >> >> Marlon >> >> >> On 6/14/13 10:54 AM, Lahiru Gunathilake wrote: >> > The one I am working is blocker ! >> > >> > Lahiru >> > >> > >> > On Fri, Jun 14, 2013 at 10:52 AM, Marlon Pierce <[email protected]> >> > wrote: >> > >> > Are the issues blockers for 0.8 release? >> > >> > >> > Marlon >> > >> > >> > On 6/14/13 10:49 AM, Lahiru Gunathilake wrote: >> >>>> Hi Marlon, >> >>>> >> >>>> I have no update about the release, since I've been working >> >>>> with Gary in Ultrascan project. I found some issues to fix >> >>>> for ultrascan, now working on them, so we have to wait until >> >>>> Gary gives the green light to do the release. >> >>>> >> >>>> WDYT ? >> >>>> >> >>>> Regards Lahiru >> >>>> >> >>>> >> >>>> On Fri, Jun 14, 2013 at 10:44 AM, Marlon Pierce >> >>>> <[email protected]> wrote: >> >>>> >> >>>> What's the status of the 0.8 release? >> >>>> >> >>>> >> >>>> Marlon >> >>>> >> >>>> >> >>>> On 5/31/13 11:53 AM, Saminda Wijeratne wrote: >> >>>>>>> On Fri, May 31, 2013 at 11:31 AM, Amila Jayasekara >> >>>>>>> <[email protected]>wrote: >> >>>>>>> >> >>>>>>>> On Fri, May 31, 2013 at 11:26 AM, Saminda Wijeratne >> >>>>>>>> <[email protected] >> >>>>>>>>> wrote: >> >>>>>>>> >> >>>>>>>>> Latest update on AiravataAPI updates for 0.8 >> >>>>>>>>> version >> >>>>>>>>> >> >>>>>>>>> 1. *Error message from the immediate exception used >> >>>>>>>>> as the error >> >>>>>>>> message >> >>>>>>>>> for the AiravataAPInvocationException*. Earlier it >> >>>>>>>>> was just "Error invoking API". *This change was >> >>>>>>>>> made so that the users of the API can show the >> >>>>>>>>> error from the exception without having to dig in >> >>>>>>>>> to the >> >>>>>>>> inner >> >>>>>>>>> exception to get a better error message.* 2. >> >>>>>>>>> *Saving and retrieving experiment execution error >> >>>>>>>>> details*. Earlier we were not persisting any >> >>>>>>>>> execution error details of the workflow. Error >> >>>>>>>>> details are only available only if the developers >> >>>>>>>>> are monitoring experiment at the time of error >> >>>>>>>>> occurs. *This is not feasible if users would only >> >>>>>>>>> want to go through the error details later without >> >>>>>>>>> doing monitoring >> >>>>>>>> from >> >>>>>>>>> the beginning. Therefore we are now persisting the >> >>>>>>>>> error details (not >> >>>>>>>> as >> >>>>>>>>> part of provenance data) of executing workflows >> >>>>>>>>> (i.e. experiments) in >> >>>>>>>> to >> >>>>>>>>> the registry and also retrieving from registry >> >>>>>>>>> through Airavata API (which calls the Registry >> >>>>>>>>> API). These functions are available in the >> >>>>>>>>> "ExecutionManager" of the AiravataAPI and >> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI. * 3. >> >>>>>>>>> Persisting & retrieving GFac job data. There are >> >>>>>>>>> many benefits of saving the GRAM data, EC2 data >> >>>>>>>>> etc. such as for debugging, analysis >> >>>>>>>> etc. >> >>>>>>>>> Currently these data are available only through >> >>>>>>>>> the message notifications. *It is not feasible for >> >>>>>>>>> the same reasons mentioned for API change for >> >>>>>>>>> saving error data. Hence the introduction of >> >>>>>>>>> functions in the API to save & retrieve GFac job >> >>>>>>>>> data.** These functions are available as >> >>>>>>>>> get/setApplicationJob***(...) functions in the >> >>>>>>>>> "ProvenanceManager" of the AiravataAPI and >> >>>>>>>>> "ProvenanceRegistry" in the RegistryAPI.* >> >>>>>>>>> >> >>>>>>>> >> >>>>>>>> May I ask why we need setApplicationJob****() methods >> >>>>>>>> in the API ? As far as I understood job information >> >>>>>>>> is populated by GFac and other internal components. >> >>>>>>>> So why do we need API methods to set those data ? (OR >> >>>>>>>> am I missing something ?) >> >>>>>>>> >> >>>>>>> GFac and Internal components also use the AiravataAPI >> >>>>>>> to populate the job data in the registry. >> >>>>>>> >> >>>>>>> >> >>>>>>>> Thanks Amila >> >>>>>>>> >> >>>>>>>> >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> Regards, >> >>>>>>>>> >> >>>>>>>>> Saminda >> >>>>>>>>> >> >>>>>>>>> >> >>>>>>>>> On Tue, May 21, 2013 at 4:59 PM, Raminder Singh < >> >>>>>>>> [email protected] >> >>>>>>>>>> wrote: >> >>>>>>>>> >> >>>>>>>>>> Thanks Chathuri, I will test and let you know. >> >>>>>>>>>> >> >>>>>>>>>> Raman On May 21, 2013, at 4:55 PM, Chathuri >> >>>>>>>>>> Wimalasena wrote: >> >>>>>>>>>> >> >>>>>>>>>>> Hi Raman, >> >>>>>>>>>>> >> >>>>>>>>>>> The method Saminda mentioned will work for your >> >>>>>>>>>>> case. I tested with >> >>>>>>>>> your >> >>>>>>>>>>> test class. I fixed deserialization issues from >> >>>>>>>>>>> REST service side. >> >>>>>>>> You >> >>>>>>>>>> will >> >>>>>>>>>>> probably have to update both server and >> >>>>>>>>>>> client. >> >>>>>>>>>>> >> >>>>>>>>>>> Regards, Chathuri >> >>>>>>>>>>> >> >>>>>>>>>>> >> >>>>>>>>>>> On Tue, May 21, 2013 at 3:04 PM, Saminda >> >>>>>>>>>>> Wijeratne < >> >>>>>>>> [email protected] >> >>>>>>>>>>> wrote: >> >>>>>>>>>>> >> >>>>>>>>>>>> So far what you are saying is true. If the >> >>>>>>>>>>>> client knows which node >> >>>>>>>>>> failed >> >>>>>>>>>>>> he/she can get the list of errors for that >> >>>>>>>>>>>> node using the getExecutionError(...) >> >>>>>>>>>>>> function. Use the "type" as ALL and leave the >> >>>>>>>>>>>> gfacJobId as null. I couldn;t think of doing >> >>>>>>>>>>>> this any other better >> >>>>>>>> way >> >>>>>>>>>> than >> >>>>>>>>>>>> introducing a whole lot of functions which >> >>>>>>>>>>>> could be more >> >>>>>>>>>> annoying/confusing >> >>>>>>>>>>>> to figure-out. I'm welcome for suggestions >> >>>>>>>>>>>> for improvements. >> >>>>>>>>>>>> >> >>>>>>>>>>>> Saminda >> >>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>>>> On Tue, May 21, 2013 at 2:36 PM, Raminder >> >>>>>>>>>>>> Singh < >> >>>>>>>>>> [email protected] >> >>>>>>>>>>>>> wrote: >> >>>>>>>>>>>> >> >>>>>>>>>>>>> Thanks Chathuri, the >> >>>>>>>>>>>>> WorkflowExecutionError works. Integrating >> >>>>>>>>>>>>> to >> >>>>>>>>>> Error >> >>>>>>>>>>>>> API brought few questions. We have >> >>>>>>>>>>>>> ErrorTypes as : >> >>>>>>>>>>>> WorkflowExecutionError, >> >>>>>>>>>>>>> ExperimentExecutionError, >> >>>>>>>>>>>>> NodeExecutionError, >> >>>>>>>> GFacJobExecutionError, >> >>>>>>>>>>>>> ExecutionError . How do we want clients to >> >>>>>>>>>>>>> know which error type to >> >>>>>>>>>>>> query? >> >>>>>>>>>>>>> I ran a job and it failed, now i want to >> >>>>>>>>>>>>> get error. Error occurred >> >>>>>>>> at >> >>>>>>>>>>>> GFac >> >>>>>>>>>>>>> provider level and client does not have >> >>>>>>>>>>>>> any information. Do we >> >>>>>>>> expect >> >>>>>>>>>>>>> client to query all different types? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> I thing we can have is a method to get all >> >>>>>>>>>>>>> error and return the >> >>>>>>>> Error >> >>>>>>>>>>>> type >> >>>>>>>>>>>>> part of the object to help the client to >> >>>>>>>>>>>>> classify the error >> >>>>>>>> location. >> >>>>>>>>>>>>> Other suggestions? >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> Raminder >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> On May 21, 2013, at 10:57 AM, Chathuri >> >>>>>>>>>>>>> Wimalasena wrote: >> >>>>>>>>>>>>> >> >>>>>>>>>>>>>> Hi Raman, >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> WorkflowExecutionError method not >> >>>>>>>>>>>>>> returning any data issue should >> >>>>>>>> be >> >>>>>>>>>>>>> fixed >> >>>>>>>>>>>>>> now. Can you get an update and check. >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> Regards, Chathuri >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>> On Tue, May 21, 2013 at 10:02 AM, >> >>>>>>>>>>>>>> Raminder Singh >> >>>>>>>>>>>>>> <[email protected]>wrote: >> >>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Thank, these are very useful for API >> >>>>>>>>>>>>>>> users. Only thing i found >> >>>>>>>>>>>> confusing >> >>>>>>>>>>>>>>> for the user is API managers. >> >>>>>>>>>>>>>>> ProvenanceManager is the one used >> >>>>>>>> to >> >>>>>>>>>>>> get >> >>>>>>>>>>>>>>> experiment status. Users will get >> >>>>>>>>>>>>>>> FINISHED or FAILED status from >> >>>>>>>>> the >> >>>>>>>>>>>>> data >> >>>>>>>>>>>>>>> object. On failure, i was trying to >> >>>>>>>>>>>>>>> find a method to get the >> >>>>>>>> error >> >>>>>>>>>>>> data >> >>>>>>>>>>>>> but >> >>>>>>>>>>>>>>> i was not able to find as those methods >> >>>>>>>>>>>>>>> exist in >> >>>>>>>> ExecutionManager. >> >>>>>>>>>>>>>>> According to me, experiment status and >> >>>>>>>>>>>>>>> in case of failures get >> >>>>>>>>> error >> >>>>>>>>>>>>> need >> >>>>>>>>>>>>>>> to be part of one manager. May be a >> >>>>>>>>>>>>>>> ExperimentManager? Getting >> >>>>>>>>>> output >> >>>>>>>>>>>>> data >> >>>>>>>>>>>>>>> can be left as part of provenance >> >>>>>>>>>>>>>>> manages as only few client may >> >>>>>>>>> want >> >>>>>>>>>>>> to >> >>>>>>>>>>>>>>> get the data in few cases. Getting >> >>>>>>>>>>>>>>> error detail in case of error >> >>>>>>>> is >> >>>>>>>>>>>>> obvious >> >>>>>>>>>>>>>>> step. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Other thing can you please explain the >> >>>>>>>>>>>>>>> difference between >> >>>>>>>>>>>>>>> WorkflowExecutionError, >> >>>>>>>>>>>>>>> ExperimentExecutionError, >> >>>>>>>>> NodeExecutionError, >> >>>>>>>>>>>>>>> GFacExecutionError on a Wiki page for >> >>>>>>>>>>>>>>> the users. Also add a >> >>>>>>>> method >> >>>>>>>>> to >> >>>>>>>>>>>>> get >> >>>>>>>>>>>>>>> JobID as that is required to get >> >>>>>>>>>>>>>>> GFacExecutionError. >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> I tested the WorkflowExecutionError >> >>>>>>>>>>>>>>> method and its not returning >> >>>>>>>>> any >> >>>>>>>>>>>>> data. >> >>>>>>>>>>>>>>> Can you please test that as well? >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> ExperimentData data = >> >>>>>>>>>>>>>>> >> >>>>>>>> >> airavataAPI.getProvenanceManager().getExperimentData(experimentID); >> >>>>>>>>>>>>>>> >> >>>>>>>> >> >>>> >> >>>>>>>> >> > >> >>>>>>>> >> List<WorkflowExecutionError> experimentExecutionErrors = >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>> >> > >> airavataAPI.getExecutionManager().getWorkflowExecutionErrors(experimentID, >> >>>>>>>>>>>>>>> >> >>>>>>>> >> >>>> >> >>>> >> > >> > >> experimentID); >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> Thanks Raminder >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> On May 20, 2013, at 5:05 PM, Saminda >> >>>>>>>>>>>>>>> Wijeratne wrote: >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Following updates will be present >> >>>>>>>>>>>>>>>> Airavata API from 0.8 release. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> 1. *Error message from the immediate >> >>>>>>>>>>>>>>>> exception used as the error >> >>>>>>>>>>>>>>> message >> >>>>>>>>>>>>>>>> for the >> >>>>>>>>>>>>>>>> AiravataAPInvocationException*. >> >>>>>>>>>>>>>>>> Earlier it was just >> >>>>>>>> "Error >> >>>>>>>>>>>>>>>> invoking API". *This change was made >> >>>>>>>>>>>>>>>> so that the users of the >> >>>>>>>> API >> >>>>>>>>>>>> can >> >>>>>>>>>>>>>>>> show the error from the exception >> >>>>>>>>>>>>>>>> without having to dig in to >> >>>>>>>> the >> >>>>>>>>>>>>> inner >> >>>>>>>>>>>>>>>> exception to get a better error >> >>>>>>>>>>>>>>>> message.* 2. *Saving and retrieving >> >>>>>>>>>>>>>>>> experiment execution error details*. >> >>>>>>>>>>>> Earlier >> >>>>>>>>>>>>>>>> we were not persisting any execution >> >>>>>>>>>>>>>>>> error details of the >> >>>>>>>>> workflow. >> >>>>>>>>>>>>>>> Error >> >>>>>>>>>>>>>>>> details are only available only if >> >>>>>>>>>>>>>>>> the developers are monitoring >> >>>>>>>>>>>>>>> experiment >> >>>>>>>>>>>>>>>> at the time of error occurs. *This is >> >>>>>>>>>>>>>>>> not feasible if users >> >>>>>>>> would >> >>>>>>>>>>>> only >> >>>>>>>>>>>>>>>> want to go through the error details >> >>>>>>>>>>>>>>>> later without doing >> >>>>>>>>> monitoring >> >>>>>>>>>>>>>>> from >> >>>>>>>>>>>>>>>> the beginning. Therefore we are now >> >>>>>>>>>>>>>>>> persisting the error details >> >>>>>>>>>>>> (not >> >>>>>>>>>>>>>>> as >> >>>>>>>>>>>>>>>> part of provenance data) of >> >>>>>>>>>>>>>>>> executing workflows (i.e. >> >>>>>>>>> experiments)* >> >>>>>>>>>>>> in >> >>>>>>>>>>>>>>>> to the registry and also retrieving >> >>>>>>>>>>>>>>>> from registry through >> >>>>>>>> Airavata >> >>>>>>>>>>>> API >> >>>>>>>>>>>>>>>> (which calls the Registry API). >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> For now we have the functions for >> >>>>>>>>>>>>>>>> saving and retrieving errors >> >>>>>>>> in >> >>>>>>>>>> the >> >>>>>>>>>>>>>>>> ExecutionManager[1]. We need to >> >>>>>>>>>>>>>>>> decide if this is the correct >> >>>>>>>>> place >> >>>>>>>>>>>> to >> >>>>>>>>>>>>>>> put >> >>>>>>>>>>>>>>>> these functions. Your thoughts in >> >>>>>>>>>>>>>>>> this matter are most >> >>>>>>>> welcome... >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> Thanks, Saminda >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>>> 1. >> >>>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>> >> > >> https://svn.apache.org/repos/asf/airavata/trunk/modules/airavata-client/src/main/java/org/apache/airavata/client/api/ExecutionManager.java >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>>> >> >>>>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>>> >> >>>>>>>>> >> >>>>>>>> >> >>>>>>> >> >>>>> >> >>>> >> >>>> >> >>>> >> >>>> >> >> >> > >> > >> > >> > >> -----BEGIN PGP SIGNATURE----- >> Version: GnuPG/MacGPG2 v2.0.18 (Darwin) >> Comment: GPGTools - http://gpgtools.org >> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ >> >> iQEcBAEBAgAGBQJRuy8SAAoJEOEgD2XReDo53w0H/2wxRPUk2TjXqwRnCdVuSYQL >> uAA6I/TCp8j1+uugNNcCKApi3npo8FTtZJDIWGDACzpzZg/zsTOp583DldzpXZXL >> e6N2Bzb+dz5woWYG/i9/EBq585ErH/e9ieDgRP0MfrryjIoNj+AKcCtutIWmfjHt >> 1E3MhpXO0yfYxiUdiYtH1nK/O5qJHhVc7foZj+Q5XiCza9Z8p3BPcKY9l5AUWMUE >> N/4HZ5mat78bJriz9qYejscVDDadfagkYAYoTQ3srKlJdXBloeL4Yhw7kE4P5gU6 >> MVFqlRIC18Fj5Ugv0bj1rMUMw7vPWiQe7fTGG2hYyCGTfsu81r9V1/3FQtyRkY0= >> =roFX >> -----END PGP SIGNATURE----- >> > > > > -- > System Analyst Programmer > PTI Lab > Indiana University > -- System Analyst Programmer PTI Lab Indiana University
