Sorry the doc didn¹t answer your question. So yes, you can include
'workflow_action' in the filter.app.types. FYI, When you also enable
SLAService on top, workflow_action gets added automatically because SLA
can be applied at that level too. But SLA and Job Notifications can be
used separate features and its not a must that you should configure both.
So the complete set of app.types would be:
<property>
<name>oozie.service.EventHandlerService.filter.app.types</name>
<value>workflow_job, workflow_action, coordinator_job,
coordinator_action </value>
<description>
The app-types among workflow/coordinator/bundle job/action
for which events system is enabled.
</description>
</property>
And you can pick and choose which subset of types you'd like.
Oozie 4.0 is released GA in beginning of September itself. and can be
downloaded from the Releases -> 4.0.0 (latest) tab on
http://oozie.apache.org/. At Yahoo, it has been tested extensively.
Regards,
Mona
On 9/18/13 12:38 PM, "Alexey Yakubovich" <[email protected]> wrote:
>Mona,
>
>Thank you for that information, it is very interesting.
>The Job Notifications feature is very useful, but am I got it right that
>it
>is about a coordinator (and workflow) jobs? That looks exactly as a "hook"
>I am looking for. But, can you apply "oozie.service.
>EventHandlerService.filter.app.types" on the workflow action level? Sorry,
>I can't get it from the doc. link you sent me.
>
>Another question, what are the expectation about 4.0 availability? When
>beta, when production?
>
>Thank you
>Alexey
>
>
>
>On Wed, Sep 18, 2013 at 1:43 PM, Mona Chitnis <[email protected]>
>wrote:
>
>> In addition,
>>
>> 1. Oozie 4.0 has the Job Notifications feature, to work with a
>> JMS-compliant message broker like ActiveMQ. The client can implement a
>> listener and receive notifications about job start/end which can be
>> error/success and related information such as job-id, name, parent-id,
>> error-code and message. To customize which 'type' of jobs you'd like to
>> filter event notifications for, override the following setting:
>>
>> <property>
>> <name>oozie.service.EventHandlerService.filter.app.types</name>
>> <value>workflow_job, coordinator_action</value>
>> <description>
>> The app-types among workflow/coordinator/bundle job/action
>>for
>> which
>> for which events system is enabled.
>> </description>
>> </property>
>>
>> 2. Oozie 4.0 has additional tables for SLA information. You can invoke
>>the
>> RESTful API to get info, without needing any internal DB API.
>>
>> Refer to the documentation about 'Accessing SLA info' -
>>
>>http://oozie.apache.org/docs/4.0.0/DG_SLAMonitoring.html#Accessing_SLA_In
>>fo
>> rmation
>>
>> To have your job tracked for SLA, you will need to surround your
>> job/action with the <sla> tags. You will find the needed tag information
>> in above twiki as well.
>>
>> Let us know if this is something you can feasibly try. If not, we can
>> further the discussion about audit-log and extensible API.
>>
>>
>> --
>> Mona Chitnis
>>
>>
>>
>> On 9/18/13 11:30 AM, "Mohammad Islam" <[email protected]> wrote:
>>
>> >What about the current options of getting job details using either of
>> >these options: CLI/Rest API/Java API?
>> >
>> >I believe most of the data should already be available. If something is
>> >missing we might need to extend that API.
>> >
>> >
>> >
>> >
>> >________________________________
>> > From: Alexey Yakubovich <[email protected]>
>> >To: [email protected]
>> >Sent: Wednesday, September 18, 2013 10:53 AM
>> >Subject: Logging from Oozie workflow
>> >
>> >
>> >Hi, this post is mostly about audit log from Oozie workflow.
>> >
>> >Suppose a silly client wants to have audit log that provides
>>immediately
>> >before / after each action:
>> >1. Name and time when action was submitted for execution
>> >2. Time when action was completed
>> >3. If completed with error, the error code, error message
>> >
>> >I see some possible ways to do it, but neither I would like.
>> >
>> >1. You can get all those data (and more) from the Oozie database
>>(tables
>> >SLA_EVENTS, WF_JOBS, WF_ACTIONS ...), if enable SLA.
>> > 1.1. Cons: Oozie database is not part of ... public API
>> > 1.2. Cons: The only reasonable way to do it (as I see it) - in
>>special
>> >java action(s) at the end of workflow. So if workflow by any way
>>doesn't
>> >get to that action, no audit logging happens
>> >
>> >2. Override the class that write data to Oozie SLA tables and made it
>>also
>> >to send some data into some other sink.
>> > 2. Cons: That seems not so transparent (clean), that class can be
>> >responsible for more than it seems, and is called often. So writing
>>from
>> >if
>> >to e.g. other (audit) database seems a bit rood to Oozie.
>> >
>> >3. Override launchers for all (most) actions, as it suggested in
>> >
>>
>>http://svn.apache.org/repos/asf/oozie/trunk/examples/src/main/apps/custom
>>-
>> >main/workflow.xml
>> > 3.1. Cons: But it also not that obvious (clean), and makes you
>> >implementation dependent; what if in next version e.g.
>> >org.apache.oozie.action.hadoop.PigMain change representation of action
>> >name
>> >(actionXML), error code, error message, etc
>> > 3.2. Cons: if code added to overridden launcher not "simple and
>> >light",
>> >that can hinder Oozie
>> >
>> >4. After every Oozie action node add the logging node.
>> > But that looks ...simple clumsy
>> >
>> >So, if I am not completely lost, it probably has sense to add some "log
>> >hooks" to actions. And to extend public API to express it.
>> >
>> >If that really what should be done to support audit logging for
>>workflow,
>> >I
>> >am willing to try to implement it after agreement on design.
>> >
>> >Thank you
>> >Alexey
>>
>>
