In addition,
1. Oozie 4.0 has the Job Notifications feature, to work with a
JMS-compliant message broker like ActiveMQ. The client can implement a
listener and receive notifications about job start/end which can be
error/success and related information such as job-id, name, parent-id,
error-code and message. To customize which 'type' of jobs you'd like to
filter event notifications for, override the following setting:
<property>
<name>oozie.service.EventHandlerService.filter.app.types</name>
<value>workflow_job, coordinator_action</value>
<description>
The app-types among workflow/coordinator/bundle job/action for
which
for which events system is enabled.
</description>
</property>
2. Oozie 4.0 has additional tables for SLA information. You can invoke the
RESTful API to get info, without needing any internal DB API.
Refer to the documentation about 'Accessing SLA info' -
http://oozie.apache.org/docs/4.0.0/DG_SLAMonitoring.html#Accessing_SLA_Info
rmation
To have your job tracked for SLA, you will need to surround your
job/action with the <sla> tags. You will find the needed tag information
in above twiki as well.
Let us know if this is something you can feasibly try. If not, we can
further the discussion about audit-log and extensible API.
--
Mona Chitnis
On 9/18/13 11:30 AM, "Mohammad Islam" <[email protected]> wrote:
>What about the current options of getting job details using either of
>these options: CLI/Rest API/Java API?
>
>I believe most of the data should already be available. If something is
>missing we might need to extend that API.
>
>
>
>
>________________________________
> From: Alexey Yakubovich <[email protected]>
>To: [email protected]
>Sent: Wednesday, September 18, 2013 10:53 AM
>Subject: Logging from Oozie workflow
>
>
>Hi, this post is mostly about audit log from Oozie workflow.
>
>Suppose a silly client wants to have audit log that provides immediately
>before / after each action:
>1. Name and time when action was submitted for execution
>2. Time when action was completed
>3. If completed with error, the error code, error message
>
>I see some possible ways to do it, but neither I would like.
>
>1. You can get all those data (and more) from the Oozie database (tables
>SLA_EVENTS, WF_JOBS, WF_ACTIONS ...), if enable SLA.
> 1.1. Cons: Oozie database is not part of ... public API
> 1.2. Cons: The only reasonable way to do it (as I see it) - in special
>java action(s) at the end of workflow. So if workflow by any way doesn't
>get to that action, no audit logging happens
>
>2. Override the class that write data to Oozie SLA tables and made it also
>to send some data into some other sink.
> 2. Cons: That seems not so transparent (clean), that class can be
>responsible for more than it seems, and is called often. So writing from
>if
>to e.g. other (audit) database seems a bit rood to Oozie.
>
>3. Override launchers for all (most) actions, as it suggested in
>http://svn.apache.org/repos/asf/oozie/trunk/examples/src/main/apps/custom-
>main/workflow.xml
> 3.1. Cons: But it also not that obvious (clean), and makes you
>implementation dependent; what if in next version e.g.
>org.apache.oozie.action.hadoop.PigMain change representation of action
>name
>(actionXML), error code, error message, etc
> 3.2. Cons: if code added to overridden launcher not "simple and
>light",
>that can hinder Oozie
>
>4. After every Oozie action node add the logging node.
> But that looks ...simple clumsy
>
>So, if I am not completely lost, it probably has sense to add some "log
>hooks" to actions. And to extend public API to express it.
>
>If that really what should be done to support audit logging for workflow,
>I
>am willing to try to implement it after agreement on design.
>
>Thank you
>Alexey
