[
https://issues.apache.org/jira/browse/OOZIE-848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13404322#comment-13404322
]
Mona Chitnis commented on OOZIE-848:
------------------------------------
Some comments regarding implementation..
1. There can be more than one bundle job (i.e. multiple bundle IDs) RUNNING
with the same bundle 'Name' (say when some data pipeline malfunctioned and the
user submitted the same bundle a second time). In this case, if the bulk query
is executed against the bundle name, there will be unnecessary duplication of
coordinator action records returned for those bundle copies. The user would
rather be alerted of such an erroneous condition when detected -> Fail-Fast
mechanism. One way of doing this is issuing an extra query in the beginning to
check for COUNT of bundle IDs against the bundle Name. Is there a better way to
have some uniqueness check on bundle in the main query for records itself?
2. Getting the ultimately complete solution to monitor realtime and be provided
with the whole data object at each level, is not as imminent as having atleast
the 'ability' to get the specific workflow IDs from bundle level that are
failing or slowing down the system. This will alleviate Oozie server memory
load to some extent.
3. Is it reasonable to execute a join operation between coordinator job and
action table using JPAExecutor query? If not, then we offload the
responsibility to Oozie server to execute multiple queries - a) to go down to
coordinator jobs from bundle and b) use the coordinator job ids as range
specifier in second query to coordinator action table
Please share your comments about the above implementation choices.
Regards,
Mona
> Bulk Monitoring API - Consolidated view of jobs
> -----------------------------------------------
>
> Key: OOZIE-848
> URL: https://issues.apache.org/jira/browse/OOZIE-848
> Project: Oozie
> Issue Type: New Feature
> Components: bundle, coordinator, workflow
> Affects Versions: trunk
> Reporter: Mona Chitnis
> Assignee: Mona Chitnis
> Attachments: API_design_choices.pdf, Design_draft_bulk_monitoring.pdf
>
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> This feature is to enable getting the status and statistics of all of the
> coordinators and coordinator actions matching a filter for a specific
> pipeline (bundle) so the user can quickly determine if there are problems and
> take corrective action.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
