OOZIE-130: build workflow progress information in Oozie
-------------------------------------------------------
Key: OOZIE-547
URL: https://issues.apache.org/jira/browse/OOZIE-547
Project: Oozie
Issue Type: New Feature
Reporter: Hadoop QA
Assignee: Roman Shaposhnik
For a user, knowing progress of her workflow is always desirable. This ticket
is to introduce this support to Oozie.
I know it's a hard problem. For my initial effort, I plan to start with simple
workflows that do not contain decision nodes or fork/join nodes, i.e., chain
type workflows. I plan to use percentage of finished actions as the overall wf
progress estimate.
Going forward we can improve the estimation by:
1) handle general workflows that contain decision, fork/join nodes;
2) incorporate the action level progress into wf level progress estimation to
make the estimate better. To be more specific:
In the case of "opaque" actions like pig/hive/jaql where the status can only be
0% or 100% (or failure) we plug that value into the overall DAG status of
0-100%. If a DAG had say 4 opaque actions, the progress would move in discrete
steps 0, 25, 50, 75, 100%. For the m/r actions where the JobTracker
gives values between 0-100% for an action then the overall progress will be
smoother. We can do same thing for pig/hive/jaql actions as well if they expose
their own progress info.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
