Author: buildbot
Date: Tue Jul 23 07:58:39 2013
New Revision: 870702
Log:
Staging update by buildbot for sling
Removed:
websites/staging/sling/trunk/content/site/apache-sling-eventing-and-job-handling.html
Modified:
websites/staging/sling/trunk/content/ (props changed)
websites/staging/sling/trunk/content/documentation/bundles/apache-sling-eventing-and-job-handling.html
websites/staging/sling/trunk/content/site/.htaccess
Propchange: websites/staging/sling/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Tue Jul 23 07:58:39 2013
@@ -1 +1 @@
-1505908
+1505925
Modified:
websites/staging/sling/trunk/content/documentation/bundles/apache-sling-eventing-and-job-handling.html
==============================================================================
---
websites/staging/sling/trunk/content/documentation/bundles/apache-sling-eventing-and-job-handling.html
(original)
+++
websites/staging/sling/trunk/content/documentation/bundles/apache-sling-eventing-and-job-handling.html
Tue Jul 23 07:58:39 2013
@@ -83,35 +83,80 @@
</div>
- <div class="tip">
- This page is a translated version of <a
href="/site/apache-sling-eventing-and-job-handling.html"
target="sling_cwiki">/site/apache-sling-eventing-and-job-handling.html</a>. In
case of
- doubt you might want to refer to the old page.
- </div>
-
<h1>Apache Sling Eventing and Job Handling</h1>
- <p><em>NOTE: This documentation is work in progress!</em></p>
-<h2 id="overview">Overview</h2>
-<p>The Apache Sling Event Support bundle provides interesting services for
advanced event handling and job processing. While this bundle leverages the
OSGi EventAdmin, it provides a very powerful support for so called jobs: a job
is a task which has to be performed by a component - the Sling job handling
ensures that exactly one component performs this task.</p>
+ <h2 id="overview">Overview</h2>
+<p>The Apache Sling Event Support bundle adds additional features to the OSGi
Event Admin and for distributed event processing.</p>
+<p>The bundle provides the following features</p>
+<ul>
+<li><a href="#jobs-guarantee-of-processing">Jobs</a></li>
+<li><a href="#distributed-events">Distributed Events</a></li>
+<li><a href="#sending-scheduled-events">Scheduled Events</a></li>
+</ul>
<p>To get some hands on code, you can refer to the following tutorials:
<em> <a
href="/documentation/tutorials-how-tos/how-to-manage-events-in-sling.html">How
to Manage Events in Sling</a>
</em> <a
href="/documentation/bundles/scheduler-service-commons-scheduler.html">Scheduler
Service (commons scheduler)</a></p>
-<p>The Sling Event Supports adds the following services:
-<em> <a href="#jobs-guarantee-of-processing">Jobs</a>
-</em> <a href="#distributed-events">Distributed Events</a>
-* <a href="#sending-scheduled-events">Scheduled Events</a></p>
<h2 id="jobs-guarantee-of-processing">Jobs (Guarantee of Processing)</h2>
<p>In general, the eventing mechanism (OSGi EventAdmin) has no knowledge about
the contents of an event. Therefore, it can't decide if an event is important
and should be processed by someone. As the event mechanism is a "fire event and
forget about it" algorithm, there is no way for an event admin to tell if
someone has really processed the event. Processing of an event could fail, the
server or bundle could be stopped etc.</p>
-<p>On the other hand, there are use cases where the guarantee of processing a
job is a must and usually this comes with the requirement of processing this
job exactly once. Typical examples are sending notification emails (or sms) or
post processing of content (like thumbnail generation of images or
documents).</p>
-<p>The Sling Event Support adds the notion of a job to the OSGi EventAdmin. A
job is a special OSGi event that someone has to process (do the job). The job
event has a special topic <em>org/apache/sling/event/job</em> to indicate that
the event contains a job. These job events are consumed by the Sling Job
Handler - it ensures that someone does the job! To support different jobs and
different processors of such jobs, the real topic of the event is stored in the
<em>event.job.topic</em> property of the original event. When a job event
(event with the topic <em>org/apache/sling/event/job</em>) is received, a new
event with the topic from the property <em>event.job.topic</em> is fired
(Firing this event comes of course with a set of rules and constraints
explained below).</p>
+<p>On the other hand, there are use cases where the guarantee of processing is
a must and usually this comes with the requirement of processing exactly once.
Typical examples are sending notification emails (or sms), post processing of
content (like thumbnail generation of images or documents), workflow steps
etc.</p>
+<p>The Sling Event Support adds the notion of a job. A job is a special event
that has to be processed exactly once. While older versions of the job handling
were based on sending and receiving events through the OSGi event admin, newer
versions provide enhanced support through special Java interface. This approach
is preferred over the still supported but deprecated event admin way.</p>
+<p>A job consists of two parts, the job topic describing the nature of the job
and the payload which is a key value map of serializable objects. A client can
initiate a job by calling the <em>JobManager.addJob</em> method:</p>
+<div class="codehilite"><pre> <span class="n">import</span> <span
class="n">org</span><span class="p">.</span><span class="n">apache</span><span
class="p">.</span><span class="n">sling</span><span class="p">.</span><span
class="n">jobs</span><span class="p">.</span><span
class="n">JobManager</span><span class="p">;</span>
+ <span class="n">import</span> <span class="n">org</span><span
class="p">.</span><span class="n">apache</span><span class="p">.</span><span
class="n">felix</span><span class="p">.</span><span class="n">scr</span><span
class="p">.</span><span class="n">annotations</span><span
class="p">.</span><span class="n">Component</span><span class="p">;</span>
+ <span class="n">import</span> <span class="n">org</span><span
class="p">.</span><span class="n">apache</span><span class="p">.</span><span
class="n">felix</span><span class="p">.</span><span class="n">scr</span><span
class="p">.</span><span class="n">annotations</span><span
class="p">.</span><span class="n">Reference</span><span class="p">;</span>
+ <span class="n">import</span> <span class="n">java</span><span
class="p">.</span><span class="n">util</span><span class="p">.</span><span
class="n">Map</span><span class="p">;</span>
+ <span class="n">import</span> <span class="n">java</span><span
class="p">.</span><span class="n">util</span><span class="p">.</span><span
class="n">HashMap</span><span class="p">;</span>
+
+ <span class="p">@</span><span class="n">Component</span>
+ <span class="n">public</span> <span class="n">class</span> <span
class="n">MyComponent</span> <span class="p">{</span>
+
+ <span class="p">@</span><span class="n">Reference</span>
+ <span class="n">private</span> <span class="n">JobManager</span> <span
class="n">jobManager</span><span class="p">;</span>
+
+ <span class="n">public</span> <span class="n">void</span> <span
class="n">startJob</span><span class="p">()</span> <span class="p">{</span>
+ <span class="n">final</span> <span class="n">Map</span><span
class="o"><</span><span class="n">String</span><span class="p">,</span>
<span class="n">Object</span><span class="o">></span> <span
class="n">props</span> <span class="p">=</span> <span class="n">new</span>
<span class="n">HashMap</span><span class="o"><</span><span
class="n">String</span><span class="p">,</span> <span
class="n">Object</span><span class="o">></span><span class="p">();</span>
+ <span class="n">props</span><span class="p">.</span><span
class="n">put</span><span class="p">(</span>"<span
class="n">item1</span>"<span class="p">,</span> "<span
class="o">/</span><span class="n">something</span>"<span
class="p">);</span>
+ <span class="n">props</span><span class="p">.</span><span
class="n">put</span><span class="p">(</span>"<span
class="n">count</span>"<span class="p">,</span> 5<span class="p">);</span>
+
+ <span class="n">jobManager</span><span class="p">.</span><span
class="n">addJob</span><span class="p">(</span>"<span
class="n">my</span><span class="o">/</span><span class="n">special</span><span
class="o">/</span><span class="n">jobtopic</span>"<span class="p">,</span>
<span class="n">null</span><span class="p">,</span> <span
class="n">props</span><span class="p">);</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+</pre></div>
+
+
+<p>The job topic follows the conventions for the topic of an OSGi event. All
objects in the payload must be serializable and publically available (exported
by a bundle). This is required as the job is persisted and unmarshalled before
processing.</p>
+<p>As soon as the method returns from the job manager, the job is persisted
and the job manager ensures that this job will be processed exactly once.</p>
+<h3 id="job-consumers">Job Consumers</h3>
+<p>A job consumer is a service consuming and processing a job. It registers
itself as an OSGi service together with a property defining which topics this
consumer can process:</p>
+<div class="codehilite"><pre> <span class="n">import</span> <span
class="n">org</span><span class="p">.</span><span class="n">apache</span><span
class="p">.</span><span class="n">felix</span><span class="p">.</span><span
class="n">scr</span><span class="p">.</span><span
class="n">annotations</span><span class="p">.</span><span
class="n">Component</span><span class="p">;</span>
+ <span class="n">import</span> <span class="n">org</span><span
class="p">.</span><span class="n">apache</span><span class="p">.</span><span
class="n">felix</span><span class="p">.</span><span class="n">scr</span><span
class="p">.</span><span class="n">annotations</span><span
class="p">.</span><span class="n">Service</span><span class="p">;</span>
+ <span class="n">import</span> <span class="n">org</span><span
class="p">.</span><span class="n">apache</span><span class="p">.</span><span
class="n">sling</span><span class="p">.</span><span class="n">event</span><span
class="p">.</span><span class="n">jobs</span><span class="p">.</span><span
class="n">Job</span><span class="p">;</span>
+ <span class="n">import</span> <span class="n">org</span><span
class="p">.</span><span class="n">apache</span><span class="p">.</span><span
class="n">sling</span><span class="p">.</span><span class="n">event</span><span
class="p">.</span><span class="n">jobs</span><span class="p">.</span><span
class="n">consumer</span><span class="p">.</span><span
class="n">JobConsumer</span><span class="p">;</span>
+
+ <span class="p">@</span><span class="n">Component</span>
+ <span class="p">@</span><span class="n">Service</span><span
class="p">(</span><span class="n">value</span><span class="p">={</span><span
class="n">JobConsumer</span><span class="p">.</span><span
class="n">class</span><span class="p">})</span>
+ <span class="p">@</span><span class="n">Property</span><span
class="p">(</span><span class="n">name</span><span class="p">=</span><span
class="n">JobConsumer</span><span class="p">.</span><span
class="n">PROPERTY_TOPICS</span><span class="p">,</span> <span
class="n">value</span><span class="p">=</span>"<span
class="n">my</span><span class="o">/</span><span class="n">special</span><span
class="o">/</span><span class="n">jobtopic</span>"<span class="p">,)</span>
+ <span class="n">public</span> <span class="n">class</span> <span
class="n">MyJobConsumer</span> <span class="n">implements</span> <span
class="n">JobConsumer</span> <span class="p">{</span>
+
+ <span class="n">public</span> <span class="n">JobResult</span> <span
class="n">process</span><span class="p">(</span><span class="n">final</span>
<span class="n">Job</span> <span class="n">job</span><span class="p">)</span>
<span class="p">{</span>
+ <span class="o">//</span> <span class="n">process</span> <span
class="n">the</span> <span class="n">job</span> <span class="n">and</span>
<span class="k">return</span> <span class="n">the</span> <span
class="n">result</span>
+ <span class="k">return</span> <span
class="n">JobResult</span><span class="p">.</span><span
class="n">OK</span><span class="p">;</span>
+ <span class="p">}</span>
+ <span class="p">}</span>
+</pre></div>
+
+
+<p>The <em>Job</em> interface allows to query the topic, the payload and
additional information about the current job. The consumer can either return
<em>JobResult.OK</em> indicating that the job has been processed,
<em>JobResult.FAILED</em> indicating the processing failed, but can be retried
or <em>JobResult.CANCEL</em> the processing has failed permanently.</p>
+<h3 id="deprecated-eventadmin-based-solution">Deprecated EventAdmin based
Solution</h3>
+<p>In previous versions starting and processing of jobs was based on the OSGi
EventAdmin. While this approach is still supported for compatibility, it's
deprecated and should be replaced with the above mentioned usage of interfaces
and services.</p>
+<p>However, if you want to use the old way, starting a job means sending an
OSGi event to the OSGi EventAdmin. The job event has a special topic
<em>org/apache/sling/event/job</em> to indicate that the event contains a job.
These job events are consumed by the Sling Job Handler - it ensures that
someone does the job. The real topic of the job is stored in the
<em>event.job.topic</em> property of the original event. When a job event
(event with the topic <em>org/apache/sling/event/job</em>) is received, a new
event with the topic from the property <em>event.job.topic</em> is fired
(Firing this event comes of course with a set of rules and constraints
explained below).</p>
<p>In order to distinguish a job which occured twice and a job which is
generated "at the same time" on several nodes, each job can be uniquely
identified by its topic (property <em>event.job.topic</em>) and an optional job
name, the <em>event.job.id</em> property. It is up to the client creating the
event to ensure that the <em>event.job.id</em> property is unqiue <em>and</em>
identical on all application nodes. If the job name is not provided for the
job, then it is up to the client to ensure that the job event is only fired
once. Usually for jobs generated based on user interaction, a unique job name
is not required as the job is only created through the user interaction.</p>
-<h3 id="job-processors">Job Processors</h3>
<p>A job processor is a service consuming and processing a job. It listens for
OSGi events with the job topic. The OSGi EventAdmin usually comes with a
timeout for event handlers. An event handler must consume an OSGi event as fast
as possible otherwise the handler might get a timeout and get blacklisted.
Therefore a job processor should never directly process the job in the event
handler method, but do this async.</p>
<p>In addition the Sling Job Handler needs to get notified if someone is
processing a job and when someone has finished processing this job.</p>
<p>To make implementing such a job processor easier, there is the
<em>JobUtil</em> utility class along with the <em>JobProcessor</em> interface.
The <em>JobUtil</em> class has a helper method for this: <em>processJob(Event,
JobProcessor)</em>. The job processor must implement the <em>JobProcessor</em>
interface which consists of a single <em>process(Event)</em> method. When the
event handler receives a job event through the OSGi EventAdmin, it calls
<em>JobUtil.processJob(event, this)</em> and returns. This utility method takes
care to notify the Sling Job Handler that someone is processing the job. Then
the <em>process(Event)</em> method of the job processor is called in the
background and when it returns, the Sling Job Handler is notified that the job
is completed (or processing failed).</p>
<p>If the job processor wants to do the background processing by itself or
does not need background processing at all, it must signal starting and
completition of the job by call <em>JobUtil.acknowledgeJob(Event),
</em>JobUtil.finishedJob(event)<em> or </em>JobUtil.rescheduleJob(Event).</p>
-<h3 id="processing-of-jobs">Processing of Jobs</h3>
-<p>Incoming jobs are first persisted in the repository (for failover etc.) and
then a job is put into a processing queue. There are different types of queues
defining how the jobs are processed (one after the other, in parallel etc.).</p>
+<h3 id="job-handling">Job Handling</h3>
+<p>New jobs are first persisted in the resource tree (for failover etc.), then
the job is distributed to an instance responsible for processing the job and on
that instance the job is put into a processing queue. There are different types
of queues defining how the jobs are processed (one after the other, in parallel
etc.).</p>
<p>For managing queues, the Sling Job Handler uses the OSGi ConfigAdmin - it
is possible to configure one or more queue configurations through the
ConfigAdmin. One way of creating and configuring such configurations is the
Apache Felix WebConsole.</p>
<h4 id="queue-configurations">Queue Configurations</h4>
<p>A queue configuration can have the following properties:</p>
@@ -123,8 +168,6 @@
| </em>queue.retries<em> | How often should the job be retried. -1 for
endless retries. |
| </em>queue.retrydelay<em> | The waiting time in milliseconds between
job retries. |
| </em>queue.priority<em> | The thread priority: NORM, MIN, or MAX |
-| </em>queue.runlocal<em> | Should the jobs only be processed on the
cluster node they have been created? |
-| </em>queue.applicationids<em> | Optional list of application (cluster
node) ids. If configured, these jobs are only processed on this application
node.|
| </em>service.ranking* | A ranking for this configuration.|</p>
<p>The configurations are processed in order of their service ranking. The
first matching queue configuration is used for the job.</p>
<h4 id="ordered-queues">Ordered Queues</h4>
@@ -134,120 +177,37 @@
<h4 id="topic-round-robin-queues">Topic-Round-Robin Queues</h4>
<p>The jobs are processed in parallel. Scheduling of the jobs is based on the
topic of the jobs. These are started by doing round-robin on the available
topics.</p>
<h4 id="ignoring-queues">Ignoring Queues</h4>
-<p>A queue of type <em>ignoring</em> ignores this job. The job is persisted
but not processed. This can be used to delay processing of some jobs. With a
changed configuration and a restart of the Sling Job Handler the ignored jobs
can be processed at a later time.</p>
+<p>A queue of type <em>ignoring</em> ignores this job. The job is persisted
but not processed. This can be used to delay processing of some jobs. With a
changed configuration, the ignored jobs can be processed at a later time.</p>
<h4 id="dropping-queues">Dropping Queues</h4>
<p>A queue of type <em>drop</em> is dropping a job - which means it is not
processed at all and directly discarded.</p>
-<h3 id="persistence">Persistence</h3>
-<p>The job event handler listens for all job events (all events with the topic
<em>org/apache/sling/event/job</em>) and will as a first step persist those
events in the JCR repository. All job events are stored in a tree under the job
root node <em>/var/eventing/jobs</em>. Persisting the job ensures proper
handling in a clustered environment and allows failover handling after a bundle
stop or server restart. Once a job has been processed by someone, the job will
be removed from the repository.</p>
-<p>When the job event listener tries to write a job into the repository it
will check if the repository already contains a job with the given topic
<em>event.job.topic</em> and job name (property <em>event.job.id</em>). If the
event has already been written by some other application node, it's not written
again.</p>
-<p>Each job is stored as a separate node with the following properties:
-| <em>Property Name</em> | <em>Description</em> |
-| <em>event:topic</em> | The topic of the job |
-| <em>event:id</em> | The unique identifier of this job (optional).
-| <em>event:created</em> | The date and time when the event has been
created (stored in the repository)
-| <em>event:application</em> | The identifier of the node where the job was
created |
-| <em>event:properties</em> | Serialized properties |
-| <em>event:finished</em> | The date and time when the job has been
finished |
-| <em>event:processor</em> | The identifier of the node which processed the
job (after successful processing) |</p>
-<p>The failover of an application node is accomplished by locking. If a job is
locked in the repository a session scoped lock is used. If this application
node dies, the lock dies as well. Each application node observes the JCR
locking properties and therefore gets aware of unlocked event nodes with the
active flag set to true. If an application node finds such a node, it locks it,
updates the <em>event:application</em> information and processes it
accordingly. In this case the event gets the additional property
<em>org/apache/sling/job/retry</em>. </p>
-<p>Each application is periodically removing old jobs from the repository
(using the scheduler).</p>
-<h3 id="distribution-of-jobs">Distribution of Jobs</h3>
-<p>A job event is an event like any other. Therefore it is up to the client
generating the event to decide if the event should be distributed. If the event
is distributed, it will be distributed with a set <em>event.application</em> on
the remote nodes. If the job event handler receives a job with the
<em>event.application</em> property set, it will not try to write it into the
repository. It will just broadcast this event asynchronously as a ~FYI
event.</p>
-<p>If a job event is created simultanously on all application nodes, the event
will not be distributed. The application node that actually has the lock on the
stored job in the repository will clear the <em>event.application</em> when
sending the event locally. All other application nodes will use the
<em>event.application</em> stored in the repository when broadcasting the event
locally.</p>
-<h2 id="usage-patterns">Usage Patterns</h2>
-<p>Based on some usage patterns, we discuss the functionality of the eventing
mechanism.</p>
-<h3 id="sending-user-generated-events">Sending User Generated Events</h3>
-<p>If a user action results in an event, the event is only created on one
single node in the cluster. The event object is generated and delivered to the
OSGi event admin. If the <em>event.distribute</em> is not explicitly set, the
event is only distributed localled.</p>
-<p>If the <em>event.distribute</em> is the, the cluster event handler will
write the event into the repository. All nodes in the cluster observe the
repository area where all events are stored. If a new event is written into
that area, each application node will get notified. It will create the event
based on the information in the repository, clear the <em>event.distribute</em>
and publish the event.</p>
-<p>The flow can be described as follows:
-1. Client code generates event using OSGi API, if the
<em>event.distribute</em> should be set, it is using the ~EventUtil.
-1. Client code sends the event to the (local) event admin.
-1. Event admin delivers the event locally.
-1. Clustering event handler receives the event if <em>event.distribute</em> is
present
-1. # Event handler adds <em>event.application</em> and writes the event to the
repository
-1. # Remote repository observers get notified through JCR observation about
the new event. They distribute the event locally with the
<em>event.application</em> (from the node where the event occured first) and
cleared <em>event.distribute</em>.</p>
-<h3 id="processing-jcr-events">Processing JCR Events</h3>
-<p>JCR events are environment generated events and therefore are sent by the
repository to each node in the cluster. In general, it is advisable to not
built the application on the low level repository events but to use application
events. Therefore the observer of the JCR event should create an OSGi event
based on the changes in the repository. A decision has to be made if the event
should be a job or a plain event.</p>
-<p>The flow can be described as follows:
-1. Client registers for JCR observation
-1. JCR notifies the client for changes
-1. Client generates OSGi event based on the JCR events (the
<em>event.distribute</em> will not be set), it decides if it sends this event
as a job.
-1. Client code sends the event to the (local) event admin
-1. Event admin publishes the event locally
-1. The distribution event handler does not set see the event as the
<em>event.distribute</em> is not set.
-1. The job event handler gets the event if it has the job topic
-1. # The job event handler adds the <em>event.application</em> property and
tries to write the job to the repository
-1. ## If no job with the topic and <em>id</em> property is in the repository,
the event will be written and locked.
-1. ## If an event with the topic and <em>id</em> property is in the repository
then:
-1. ### If the <em>event.application</em> equals the current application node,
the event is set to active (<em>event:active</em>) in the repository again and
locked
-1. ### If the <em>event.application</em> does not equal the current
application node, the event is not distributed locally.
-1. ## If the job could be locked in the repository, the job event handler
delivers the job locally and synchronously and it unlocks the job and sets
<em>event:active</em> to false afterwards.</p>
-<h3 id="sending-scheduled-events">Sending Scheduled Events</h3>
-<p>Scheduled events are OSGi events that have been created by the environemnt.
They are generated on each application node of the cluster through an own
scheduler instance. Sending these events works the same as sending events based
on JCR events (see above).</p>
-<p>In most use cases a scheduler will send job events to ensure that exactly
one application node is processing the event.</p>
-<h3 id="receiving-osgi-events">Receiving OSGi Events</h3>
-<p>If you want to receive OSGi events, you can just follow the specification:
receive it via a custom event handler which is registered on bundle start - a
filter can be specified as a configuration property of the handler. </p>
-<p>As we follow the principle of distributing each event to every registered
handler, the handler has to decide if it will process the event. In order to
avoid multiple processing of this event in a clustered environment, the event
handler should check the <em>event.application</em> property. If it is not set,
it's a local event and the handler should process the event. If the
<em>event.application</em> is set, it's a remote event and the handler should
not process the event. This is a general rule of thumb - however, it's up to
the handler to make its decision either on <em>event.application</em> or any
other information.</p>
-<p>It is advisable to perform the local event check even in a non clustered
environment as it makes the migration to a cluster later on much easier and
there is nearly no performance overhead caused by the check.</p>
-<p>The ~EventUtil class provides an utility method
<em>isLocalEvent(Event)</em> which checks the existance of the
<em>event.application</em> property and returns <em>true</em> if it is
absend.</p>
+<h3 id="job-distributing">Job Distributing</h3>
+<p>For job distribution (= distributing the processing in a cluster), the job
handling uses the topology feature from Sling - each instance in the topology
announces the set of topics (consumers) it currently has - and this defines the
job capabilities, a mapping from an instance to the topics it can process.</p>
+<p>When a job is scheduled, the job manager uses these capabilities to find
out the set of instances which is able to process the request. If the queue
type is <em>ordered</em> then all jobs are processed by the leader of this set.
For parallel queues, the jobs are distributed equally amongst those
instance.</p>
+<p>Failover is handled by the leader: if an instance dies, the leader will
detect this through the topology framework and then redistribute jobs from the
dead instance to the available instances. Of course this takes a leader change
into account as well. In addition if the job capabilities change and this
require a reschedule of jobs, that's done by the leader as well.</p>
+<h3 id="job-creation-patterns">Job Creation Patterns</h3>
+<p>The job manager ensures that a job is processed exactly once. However, the
client code has to take care that a job is created exactly once. We'll discuss
this based on some general usage patterns:</p>
+<h4 id="jobs-based-on-user-action">Jobs based on user action</h4>
+<p>If a user action results in the creation of a job, the thread processing
the user action can directly create the job. This ensures that even in a
clustered scenario the job is created only once.</p>
+<h4 id="jobs-based-on-observation-events">Jobs based on observation /
events</h4>
+<p>If an observation event or any other OSGi event results in the creation of
a job, special care needs to be taken in a clustered installation to avoid the
job is created on all cluster instances. The easiest way to avoid this, is to
use the topology api and make sure the job is only created on the leader
instance.</p>
+<p>If that's not doable, the job handling provides the support of a unique job
name - if all instances in a cluster use the same unique job name to create the
job, the job handling detects this and will start this job just once.
Additional jobs arriving with the same name are discarded. The
<em>JobManager.addJob</em> method can be provided with this name as the second
argument.</p>
+<p>However as processing of a job name comes with some overhead, it should
only be used if there is no other way. Using the topology api with the leader
detection is the preferred way.</p>
<h2 id="distributed-events">Distributed Events</h2>
<p>In addition to the job handling, the Sling Event support adds handling for
distributed events. A distributed event is an OSGi event which is sent across
JVM boundaries to a different VM. A potential use case is to broadcast
information in a clustered environment.</p>
-<h3 id="sources-of-events">Sources of Events</h3>
-<p>When it comes to application based on Sling, there is a variety of sources
from which OSGi events can be send:
-<em> JCR observation events
-</em> Application generated events
-<em> Events from messaging systems (~JMS)
-</em> "External events"</p>
-<p>The events can either be generated inside a current user context, e.g. when
the user performs an action through the UI, or they can be out of a user
context, e.g. for schedulded events. This leads to different weights of
events.</p>
-<h3 id="weights-of-events">Weights of Events</h3>
-<p>We can distinguish two different weights of events, depending how they are
distributed in a clustered environment:</p>
-<ul>
-<li>User generated events - these events are generated directly by some user
action and are therefore started on one single node.</li>
-<li>Environment generated events (JCR events, scheduler events etc.) - these
events are generated "simultanously" on all nodes.</li>
-</ul>
-<p>External events, like incoming JMS events etc. might fall either into the
first or the second category. The receiver of such events must have the
knowledge about the weight of the event.</p>
<h3 id="basic-principles">Basic Principles</h3>
-<p>The foundation of the distributed event mechanism is to distribute each
event to every node in a clustered environment. The event distribution
mechanism has no knowledge about the intent of the event and therefore is not
able to make delivery decisions by itself. It is up to the sender to decide
what should happen, however the sender must explicitly declare an event to be
distributed. There are exceptions to "distributing everything to everywhere" as
for example framework related events (bundle stopped, installed etc.) should
not be distributed.</p>
+<p>The foundation of the distributed event mechanism is to distribute each
event to every node in a clustered environment. The event distribution
mechanism has no knowledge about the intent of the event and therefore is not
able to make delivery decisions by itself. It is up to the sender to decide
what should happen. The sender must explicitly declare an event to be
distributed as for example framework related events (bundle stopped, installed
etc.) should not be distributed.</p>
<p>The event mechanism will provide additional functionality making it easier
for event receivers to decide if they should process an event. The event
receiver can determine if the event is a local event or comming from a remote
application node. Therefore a general rule of thumb is to process events only
if they're local and just regard remote events as a FYI.</p>
-<p>The event mechanism is an <em>event</em> mechanism which should not be
confused with a <em>messaging</em> mechanism. Events are received by the event
mechanism and distributed to registered listeners. Concepts like durable
listeners, guarantee of processing etc. are not part of the event mechanism
itself. However, there is additional support for such things, like job
handling.</p>
-<p>The application should try to use application events instead of low level
JCR events whereever possible. Therefore a bridging between JCR events and the
event mechanism is required. However, a general "automatic" mapping will not be
provided. It is up to the application to develop such a mapping on a per use
case base. There might be some support to make the mapping easier.</p>
-<p>The event handling should be made as transparent to the developer as
possible. Therefore the additional code for a developer to make the eventing
working in a clustered environment etc. should be kept to a minimum (which will
hopefully reduce possible user errors).</p>
-<h3 id="distributed-events_1">Distributed Events</h3>
<p>For distributed events two properties are defined (check the
<em>EventUtil</em> class):
<em> </em>event.distribute<em> - this flag is set by the sender of an event to
give a hint if the event should be distributed across instances. For example
JCR observation based events are already distributed on all instances, so there
is no further need to distribute them. If the flag is present, the event will
be distributed. The value has currently no meaning, however the EventUtil
method should be used to add this property. If the flag is absent the event is
distributed locally only.
</em> <em>event.application</em> - An identifier for the current application
node in the cluster. This information will be used to detect if an event has
been created on different nodes. If the event has been created on the same
node, the <em>event.application</em> is missing, if it is a remote event, the
<em>event.application</em> contains the ID of the node, the event has been
initially created. Use the <em>EventUtil.isLocal(Event)</em> method to detect
if the event is a local or a distributed event.</p>
<p>While the <em>event.distribute</em> must be set by the sender of an event
(if the event should be distributed), the <em>event.application</em> property
is maintained by the event mechanism. Therefore a client sending an event
should <em>never</em> set this information by itself. This will confuse the
local event handlers and result in unexpected behaviour. On remote events the
<em>event.application</em> is set by the event distribution mechanism.</p>
<h3 id="event-distribution-across-application-nodes-cluster">Event
Distribution Across Application Nodes (Cluster)</h3>
-<p>The (local) event admin is the service distributing events locally. The
Sling Distributing Event Handler is a registered event handler that is
listening for events to be distributed. It distributes the events to remote
application notes, the JCR repository is used for distribution. The
distributing event handler writes the events into the repository, the
distributing event handlers on other application nodes get notified through
observation and then distribute the read events locally.</p>
+<p>The (local) event admin is the service distributing events locally. The
Sling Distributing Event Handler is a registered event handler that is
listening for events to be distributed. It distributes the events to remote
application notes, Sling's resource tree is used for distribution. The
distributing event handler writes the events into the resource tree, the
distributing event handlers on other application nodes get notified through
observation and then distribute the read events locally.</p>
<p>As mentioned above, the client sending an event has to mark an event to be
distributed in a cluster by setting the <em>event.distribute</em> in the event
properties (through <em>EventUtil</em>). This distribution mechanism has the
advantage that the application nodes do not need to know each other and the
distribution mechanism is independent from the used event admin
implementation.</p>
-<h3 id="storing-events-in-the-repository">Storing Events in the Repository</h3>
-<p>Distributable events are stored in the repository, the repository will have
a specific area (path) where all events are stored. </p>
-<p>Each event is stored as a separate node with the following properties:
-| <em>Property Name</em> | <em>Description</em> |
-| <em>event:topic</em> | The topic of the event |
-| <em>event:application</em> | The identifier of the application node where
the event was created |
-| <em>event:created</em> | The date and time when the event has been
created (stored in the repository)
-| <em>event:properties</em> | Serialized properties (except the
<em>event.distribute</em>, but including the <em>event.application</em>) |</p>
-<p>Each application is periodically removing old events from the repository
(using the scheduler).</p>
-<h3 id="sending-scheduled-events_1">Sending Scheduled Events</h3>
+<h2 id="sending-scheduled-events">Sending Scheduled Events</h2>
<p>Scheduled events are OSGi events that have been created by the environemnt.
They are generated on each application node of the cluster through an own
scheduler instance. Sending these events works the same as sending events based
on JCR events (see above).</p>
-<p>In most use cases a scheduler will send job events to ensure that exactly
one application node is processing the event.</p>
-<h3 id="receiving-osgi-events_1">Receiving OSGi Events</h3>
-<p>If you want to receive OSGi events, you can just follow the specification:
receive it via a custom event handler which is registered on bundle start - a
filter can be specified as a configuration property of the handler. </p>
-<p>As we follow the principle of distributing each event to every registered
handler, the handler has to decide if it will process the event. In order to
avoid multiple processing of this event in a clustered environment, the event
handler should check the <em>event.application</em> property. If it is not set,
it's a local event and the handler should process the event. If the
<em>event.application</em> is set, it's a remote event and the handler should
not process the event. This is a general rule of thumb - however, it's up to
the handler to make its decision either on <em>event.application</em> or any
other information.</p>
-<p>It is advisable to perform the local event check even in a non clustered
environment as it makes the migration to a cluster later on much easier and
there is nearly no performance overhead caused by the check.</p>
-<p>The ~EventUtil class provides an utility method
<em>isLocalEvent(Event)</em> which checks the existance of the
<em>event.application</em> property and returns <em>true</em> if it is
absend.</p>
-<h2 id="scheduler">Scheduler</h2>
-<p>Each Sling based application will contain a scheduler service (which is
based on the Quartz open source project).</p>
-<h2 id="use-cases">Use Cases</h2>
-<h3 id="post-processing-business-processes">Post Processing (Business
Processes)</h3>
-<p>A typical example for post processing (or running a business process) is
sending an email or creating thumbnails and extracting meta data from the
content (like we do in DAM), which we will discuss here.</p>
-<p>An appropriate JCR observer will be registered. This observer detects when
new content is put into the repository or when content is changed. In these
cases it creates appropriate <em>CONTENT</em>ADDED<em>,
</em>CONTENT<em>UPDATED</em> OSGi events from the JCR events. In order to
ensure that these actions get processed accordingly, the event is send as a job
(with the special job topic, the <em>topic</em> and <em>id</em> property).</p>
-<p>The event admin now delivers these jobs to the registered handlers. The job
event handler gets notified and (simplified version) sends the contained event
synchronously. One of the handlers for these events is the post processing
service in DAM. The job mechanism ensures that exactly one application node is
post processing and that the process has to be finished even if the application
node dies during execution.</p>
-<h2 id="scheduling">Scheduling</h2>
-<p>The scheduler is a service which uses the open source Quartz library. The
scheduler has methods to start jobs periodically or with a cron definition. In
addition, a service either implementing <em>java.lang.Runnable</em> or
<em>org.quartz.job</em> is started through the whiteboard pattern <em>if</em>
it either contains a configuration property <em>scheduler.expression</em> or
<em>scheduler.period</em>. The job is started with the ~PID of the service - if
the service has no PID, the configuration property <em>scheduler.name</em> must
be set.</p>
<div class="timestamp" style="margin-top: 30px; font-size: 80%;
text-align: right;">
- Rev. 1499238 by fmeschbe on Wed, 3 Jul 2013 07:39:54 +0000
+ Rev. 1505925 by cziegeler on Tue, 23 Jul 2013 07:58:25 +0000
</div>
<div class="trademarkFooter">
Apache Sling, Sling, Apache, the Apache feather logo, and the Apache
Sling project
Modified: websites/staging/sling/trunk/content/site/.htaccess
==============================================================================
--- websites/staging/sling/trunk/content/site/.htaccess (original)
+++ websites/staging/sling/trunk/content/site/.htaccess Tue Jul 23 07:58:39 2013
@@ -5,6 +5,7 @@ Redirect Permanent /site/architecture.ht
Redirect Permanent /site/adapters.html
/documentation/the-sling-engine/adapters.html
Redirect Permanent /site/apache-sling-commons-thread-pool.html
/documentation/bundles/apache-sling-commons-thread-pool.html
Redirect Permanent /site/apache-sling-community-roles-and-processes.html
/project-information/apache-sling-community-roles-and-processes.html
+Redirect Permanent /site/apache-sling-eventing-and-job-handling.html
/documentation/bundles/apache-sling-eventing-and-job-handling.html
Redirect Permanent /site/authentication.html
/documentation/the-sling-engine/authentication.html
Redirect Permanent /site/authentication-actors.html
/documentation/the-sling-engine/authentication/authentication-actors.html
Redirect Permanent /site/authentication-authenticationhandler.html
/documentation/the-sling-engine/authentication/authentication-authenticationhandler.html
