[
https://issues.apache.org/jira/browse/OODT-203?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13123718#comment-13123718
]
Chris A. Mattmann edited comment on OODT-203 at 10/9/11 4:29 PM:
-----------------------------------------------------------------
Hi Sheryl,
Thanks for your questions. Comments below:
bq. Has the NonBlockingThreadPoolWorkflowEngine implementation been updated
after the initial one? I went through the code from the checked out version
around r1156658 of OODT 0.4 and it looks just as it is described by David's
comment.
There was a NonBlockingThreadPoolWorkflowEngine. In a way you could call it a
predecessor to the Wengine branch. As I started to integrate Wengine in
OODT-215 into trunk, I decided to get rid of the existing old implementation,
and bring Wengine down in new. The stub I'm about to attach in OODT-310 that
contains the PrioritizedQueueBasedWorkflowEngine is the functional replacement.
bq. In the above comment, which higher level file manager services are you
referring for modeling the unit test?
I mean the tests in filemgr that actually instantiate an XmlRpcFileManager
service, and then ingest files into it. Similarly, we should have some tests in
the workflow manager that stand up an XmlRpcWorkflowManager service, and then
run workflows and check the expected results. Those would be great!
bq. By the PEATE example, do you mean the case described in OODT-202?
Yep. More specifically, PEATE is a Product Evaluation and Testbed Element
(PEATE) project part of the NASA/DOD/NOAA NPOESS Preparatory Project. This code
and effort originally came out of the NPP Sounder PEATE, which resides at JPL,
and their use cases.
bq. How will AbstractBaseWorkflowEngine be different from the current core
WorkflowEngine? Are you thinking of adding some new functionality to the
existing ones?
In the old design there was thinking that we could aggregate a bunch of common
functionality in an AbstractWorkflowEngine, that the subclasses (like
ThreadPoolWorkflowEngine) inherited from. I'm not sure we're headed down that
direction anymore though.
was (Author: chrismattmann):
Hi Sheryl,
Thanks for your questions. Comments below:
bq. Has the NonBlockingThreadPoolWorkflowEngine implementation been updated
after the initial one?
I went through the code from the checked out version around r1156658 of OODT
0.4 and it looks just as it is described by David's comment.
There was a NonBlockingThreadPoolWorkflowEngine. In a way you could call it a
predecessor to the Wengine branch. As I started to integrate Wengine in
OODT-215 into trunk, I decided to get rid of the existing old implementation,
and bring Wengine down in new. The stub I'm about to attach in OODT-310 that
contains the PrioritizedQueueBasedWorkflowEngine is the functional replacement.
bq. In the above comment, which higher level file manager services are you
referring for modeling the unit test?
I mean the tests in filemgr that actually instantiate an XmlRpcFileManager
service, and then ingest files into it. Similarly, we should have some tests in
the workflow manager that stand up an XmlRpcWorkflowManager service, and then
run workflows and check the expected results. Those would be great!
bq. By the PEATE example, do you mean the case described in OODT-202?
Yep. More specifically, PEATE is a Product Evaluation and Testbed Element
(PEATE) project part of the NASA/DOD/NOAA NPOESS Preparatory Project. This code
and effort originally came out of the NPP Sounder PEATE, which resides at JPL,
and their use cases.
bq. How will AbstractBaseWorkflowEngine be different from the current core
WorkflowEngine? Are you thinking of adding some new functionality to the
existing ones?
In the old design there was thinking that we could aggregate a bunch of common
functionality in an AbstractWorkflowEngine, that the subclasses (like
ThreadPoolWorkflowEngine) inherited from. I'm not sure we're headed down that
direction anymore though.
> Create a Non-Blocking threaded implementation of the Workflow Engine
> --------------------------------------------------------------------
>
> Key: OODT-203
> URL: https://issues.apache.org/jira/browse/OODT-203
> Project: OODT
> Issue Type: Sub-task
> Components: workflow manager
> Environment: from JPL's last internal WM JIRA release
> Reporter: David Woollard
> Assignee: Chris A. Mattmann
> Fix For: 0.4
>
>
> The current implementation of the threaded workflow engine uses a thread from
> its thread pool for each of the workflows it creates. This can cause a
> problem if a significant number of workflows submitted block waiting on
> preconditions to be satisfied. Each of these paused workflows hangs on to the
> thread allocated from the thread pool, so workflows that could be run are
> blocked by workflows waiting on preconditions if the thread pool is fully
> allocated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
