Yasas and Marcus: Please see below.. On Sep 20, 2018, at 12:15 AM, Yasas Gunarathne <yasasgunarat...@gmail.com<mailto:yasasgunarat...@gmail.com>> wrote:
Hi Marcus, Thank you very much for the suggestion. In the beginning, I tried to use the current ExperimentModel to implement workflows since it has workflow related characteristics as you have mentioned. It seemed to be designed at first keeping the workflow as a primary focus including even ExperimentType.WORKFLOW. But, apart from that and the database level one-to-many relationship with processes, there is no significant support provided for workflows. I believe processes should be capable of executing independently at their level of abstraction. But, in the current architecture processes execute some experiment related parts going beyond their scope. For example, saving experiment output along with process output after completing the process, which is not required for workflows. Here, submitting a message to indicate the process status should be enough. This can be implemented as an option, keeping the saving the files at the end of each step of the workflow either as default. If processes can execute independently, it doesn't need to keep experiment_id within itself in the table. Isn't it the responsibility of whatever the outer layer (Experiment/Workflow) to keep this mapping? WDYT? :) An Experiment can have multiple jobs. That way the experiment ID becomes equivalent to a workflow ID but among the processes multiple jobIDs could be associated. As you have mentioned we can keep an additional Experiment within Workflow Application to keeping the current Process execution unchanged. (Here the experiment is still executing a single application.) Is that what you meant? For each job different application could be associated and the requirement that we start with Application need to be modified. All these are assuming we keep the experimentID at the top level and provide a way to add more applications based jobs as processes below along with data staging steps. Thanks, Sudhakar. Regards On Thu, Sep 20, 2018 at 6:22 AM Christie, Marcus Aaron <machr...@iu.edu<mailto:machr...@iu.edu>> wrote: On Sep 8, 2018, at 9:11 AM, Yasas Gunarathne <yasasgunarat...@gmail.com<mailto:yasasgunarat...@gmail.com>> wrote: To represent workflow application -> process relationship, it is required to fill the experiment_id field of the process with a null and then add an entry to an intermediate table to keep the above relationship, and modify the logic a little bit to handle null experiment ids (which seems like a bad way to do it). Hi Yasas, Sorry for the late reply. Not sure if this helps, but I would think of an experiment as an execution of a workflow instead of simply being one-to-one with a process. The ExperimentModel already features a list of Processes, although recently without workflows there is generally only one Process. Maybe the workflow application can reference an Experiment that keeps the relationship with the processes for the workflow? Thanks, Marcus -- Yasas Gunarathne Undergraduate at Department of Computer Science and Engineering Faculty of Engineering - University of Moratuwa Sri Lanka LinkedIn<https://www.linkedin.com/in/yasasgunarathne/> | GitHub<https://github.com/yasgun> | Mobile : +94 77 4893616
