[
https://issues.apache.org/jira/browse/FALCON-1231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661356#comment-14661356
]
Pallavi Rao commented on FALCON-1231:
-------------------------------------
Had a quick discussion with [~sriksun] too. I will pursue Oozie JMS
notification option for 2 reasons:
1. JMS notifications are reliable
2. We get notified on all intermediate states too.
There is a flip side to this too:
1. The JMS notification is very light weight requiring us to make another call
to Oozie to get all the config related information (nominal time, name etc.).
We can try and optimize this by querying oozie only during prep and maintaining
an in-memory data structure for Workflow ID to instance ID mapping. This way,
we make only Oozie query per instance.
> Improve JobCompletionNotification Service
> -----------------------------------------
>
> Key: FALCON-1231
> URL: https://issues.apache.org/jira/browse/FALCON-1231
> Project: Falcon
> Issue Type: Sub-task
> Reporter: Pallavi Rao
> Assignee: Pallavi Rao
> Attachments: OozieJobStatusNotification.pdf
>
>
> Workflow completion is currently dependent on post processing action putting
> messages (success/failure) to a JMS queue.
> When failures occur on Hadoop / Oozie, the post processing action may not
> get executed either. In such cases, Falcon will be blind to the status of the
> workflow and re-tries will not happen.
> Explore url notification of oozie to get job completion notifications. Fall
> back on polling when workflows run beyond SLA.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
