Author: buildbot
Date: Tue Jan 31 08:00:37 2012
New Revision: 803627
Log:
Staging update by buildbot for stanbol
Modified:
websites/staging/stanbol/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.html
Modified:
websites/staging/stanbol/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.html
==============================================================================
---
websites/staging/stanbol/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.html
(original)
+++
websites/staging/stanbol/trunk/content/stanbol/docs/trunk/enhancer/executionmetadata.html
Tue Jan 31 08:00:37 2012
@@ -57,9 +57,13 @@
<div id="content">
<h1 class="title">Execution Metadata</h1>
- <p>The execution metadata are added by the <a
href="enhancementjobmanager.html">EnhancementJobManager</a> to the metadata of
the <a href="contentitem.html">ContentItem</a>. This metadata provide
information about the execution of the <a
href="chains/executionplan.html">ExecutionPlan</a> provided by the <a
href="chains/enhancementchain.html">Chain</a> and can be used by clients to get
detailed information about the enhancement process of a content item.</p>
-<p>In the case of asynchronous calls to the enhancers RESTful interface
(requests that immediately return and do not wait for the enhancement process
to complete) this information might also be useful to provide information about
the current state of the enhancement process.</p>
-<h3 id="exection_metadata_ontology">Exection Metadata Ontology</h3>
+ <p>The execution metadata provide detailed information about an
ongoing/completed enhancement process. Basically they describe how the <a
href="chains/executionplan.html">ExecutionPlan</a> provided by the <a
href="chains.enhancementchain.html">Chain</a> was executed by the <a
href="enhancementjobmanager.html">EnhancementJobManager</a>. Both the
ExecutionMetadata and the ExecutionPlan are provided with the ContentItem as an
own content part of the type MGraph with the URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";.
For users of the Stanbol Enhacer the Execution Metadata are of interest to:</p>
+<ul>
+<li>check progress of asynchronously started Enhancement Processes: Metadata
for all planed executions of engines are created as soon as an ContentItem is
parsed to the EnhancementJobManager and are updated as soon as the execution of
engines start/complete/fail.</li>
+<li>Monitor the performance of different EnhancementEngines: The Execution
Metadata provide detailed information about starting/completion time points for
engine executions.</li>
+<li>Inspect the Enhancement Process: Check if optional EnhancementEngines
where successfully executed or where skipped/failed; Validate the configured
EnhancementChain by checking the actual execution order of the
EnhancementEngines.</li>
+</ul>
+<h2 id="exection_metadata_ontology">Exection Metadata Ontology</h2>
<p>The RDFS schema used for the execution plan is defined as follows.</p>
<ul>
<li>Namespace: em :
http://stanbol.apache.org/ontology/enhancer/executionMetadata#</li>
@@ -85,6 +89,7 @@
<li><strong>em:ExecutionStatus</strong> : Class describing the status of an
EngineExecution<ul>
<li><strong>em:StatusSheduled</strong> : ExecutionStatis instance that
described that an execution is scheduled but has not yet started</li>
<li><strong>em:StatusInProgress</strong> : ExecutuinStatus instance that
describes that the execution of the linked EngineExecution is in progress</li>
+<li><strong>em:StatusSkipped</strong> : ExecutionStatus instance describing
that the execution was skipped. This can only be the case for optional
ep:ExecutionNodess</li>
<li><strong>em:StatusCompleted</strong> : ExecutionStatus instance describing
that the execution has already completed successfully</li>
<li><strong>em:StatusFailed</strong> : ExecutionStatus indicating that the
execution has failed. Typically a em:statusMessage describing the reason for
the failed execution is provided for em:Executions with that state.</li>
<li><strong>em:StatusSkiped</strong> : ExecutionStatus indicating that the
execution if an sp:ExecutionNode was skipped. This is only allowed for
execution nodes that are marked as optional. Typically also a em:statusMessage
with the reason should be provided.</li>
@@ -179,7 +184,128 @@
</pre></div>
-<p>Note that both the Execution Metadata AND the Execution Plan need to be
contained within the metadata of the <a
href="contentitem.html">ContentItem</a></p>
+<h2 id="creationmanagement_of_executionmetadata">Creation/Management of
ExecutionMetadata</h2>
+<p>This section is primarily intended for implementors of
EnhancementJobManager. However it might also provide insights for users that
want/need to monitor the state of enhancement processes as it describes what
information are added when to the Execution Metadata.</p>
+<p>When the <a href="enhancementjobmanager.html">EnhancementJobManager</a>
starts the Enhancement of an ContentItem he needs check if the <a
href="contentitem.html">ContentItem</a> already contains ExecutionMetadata in
the ContentPart with the URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";.
If this is the case it needs to initialize itself based on the pre-existing
information. If no ExecutionMetadata are present a new EnhancementProcess needs
to be created based on the parsed Chain. Difreencies between this two cases are
explained in the rolling two sub sections</p>
+<h3 id="initialization">Initialization</h3>
+<p>If no ExecutionMetadata are present within a parsed ContentItem a new
EnhancementProcess needs to be set up. This includes the following steps:</p>
+<ol>
+<li>Get the <a href="chains/executionplan.html">ExecutionPlan</a> for the
parsed enhancement <a href="chains/enhancementchain.html">Chain</a>. If no
chain is parsed the default chain need to be acquired by using the <a
href="chains/chainmanager.html">ChainManager</a>.</li>
+<li>Create the content part for the ExecutionMetadata with the <a
href="contentitem.html">ContentItem</a> and add the information of the <a
href="chains/executionplan.html">ExecutionPlan</a> to it.</li>
+<li>Create the initial ExecutionMetadata. This includes the
'em:ChainExecution' instance for the 'ep:ExecutionPlan' as well as
'em:EngineExecution' instances for all 'ep:ExecutionNode's defined by the
execution plan. All such 'em:Execution' instances MUST BE created with the
'em:ExecutionStatus' 'em:StatusSheduled'.</li>
+</ol>
+<p>The ExecutionMetadataHelper utility of the
"org.apache.stanbol.enhancer.servicesapi" module contains utility methods for
initializing execution metadata.</p>
+<h3 id="continuation">Continuation</h3>
+<p>If the parsed ContentItem does already contain ExecutionMetadata in the
content part with the URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";
the EnhancementJobManager MUST follow the following steps to continue an
EnhancementProcess.</p>
+<ol>
+<li>Check if the contained ExecutionMetadata are valid<ul>
+<li>If a 'em:ChainExecution' node is present that 'em:enhances' the parsed
ContentItem</li>
+<li>If the ExecutionPlan is included and if the value of the 'ep:chain'
property for the 'ep:ExecutionPlan' resource corresponds to the name of the
Chain parsed in the request.</li>
+</ul>
+</li>
+<li>Check the status of all 'em:Execution' instances<ul>
+<li>reset the status of 'em:Execution's that are in-progress to scheduled.</li>
+<li>TODO: here we could also retry the execution of failed 'em:Execution's</li>
+</ul>
+</li>
+</ol>
+<p>Note that with an Continuation the ExecutionPlan MUST NOT be updated. It
MUST BE also NOT checked if a Chain with the name as stored in the
ExecutionMetadata is still present. Note also that configuration changes of
EnhancementEngine will affect the continuation of the enhancement process.</p>
+<p>The ExecutionMetadataHelper utility of the
"org.apache.stanbol.enhancer.servicesapi" module contains utility methods for
reading and validating pre-existing execution metadata.</p>
+<h3 id="execution_state_management">Execution State Management</h3>
+<p>The following metadata need to be updated by the EnhancementJobManager
when:</p>
+<ul>
+<li>Enhancement process starts<ul>
+<li>set the 'em:status' of the 'em:ChainExecution' to
'em:StatusInProgress'</li>
+<li>set the 'em:started' to the current date time</li>
+</ul>
+</li>
+<li>EnhancementEngine execution starts:<ul>
+<li>set the 'em:status' of the 'em:EngineExecution' to
'em:StatusInProgress'</li>
+<li>set the 'em:started' to the current date time</li>
+</ul>
+</li>
+<li>EnhancementEngine completes<ul>
+<li>set the 'em:status' of the 'em:EngineExecution' to
'em:StatusCompleted'</li>
+<li>set the 'em:completed' to the current date time</li>
+</ul>
+</li>
+<li>Optional EnhancementEngine not available<ul>
+<li>set the 'em:status' of the 'em:EngineExecution' to 'em:StatusSkipped'</li>
+<li>set both 'em:started' and 'em:completed' to the current date time</li>
+</ul>
+</li>
+<li>Optional EnhancementEngine failed<ul>
+<li>set the 'em:status' of the 'em:EngineExecution' to 'em:StatusFailed'</li>
+<li>set the 'em:completed' to the current date time</li>
+</ul>
+</li>
+<li>Required EnhancementEngine failed or not available<ul>
+<li>set the 'em:status' of the 'em:EngineExecution' to 'em:StatusFailed'</li>
+<li>set the 'em:status' of the 'em:ChainExecution' to 'em:StatusFailed'</li>
+<li>set the 'em:completed' of both the engine and the chain execution to the
current date time</li>
+</ul>
+</li>
+<li>Enhancement process completes<ul>
+<li>set the 'em:status' of the 'em:ChainExecution' to 'em:StatusCompleted'</li>
+<li>set the 'em:completed' to the current date time</li>
+</ul>
+</li>
+<li>Internal error in the EnhancementJobManager implementation<ul>
+<li>set the 'em:status' of the 'em:ChainExecution' to 'em:StatusFailed'</li>
+<li>do not set any 'em:EngineExecution' to failed.</li>
+<li>set the 'em:completed' value of the 'em:ChainExecution' to the current
date time</li>
+</ul>
+</li>
+</ul>
+<p>The ExecutionMetadataHelper utility of the
"org.apache.stanbol.enhancer.servicesapi" module contains utility methods to
preform state transitions on 'em:Execution' instances.</p>
+<h2 id="using_executionmetadata">Using ExecutionMetadata</h2>
+<p>This Section provides some Examples on how to access and retrieve
information from the ExecutionMetadata.</p>
+<h3 id="accessing_executionmetadata">Accessing ExecutionMetadata</h3>
+<p>The ExecutionMetadata and the <a
href="chains/executionplan.html">ExecutionPlan</a> are stored in a content part
with with URI
"http://stanbol.apache.org/ontology/enhancer/executionMetadata#ChainExecution";
with the <a href="contentitem.html">ContentItem</a>. The following code segment
can be used to retrieve the RDF graph with the ExecutionMetadata:</p>
+<div class="codehilite"><pre><span class="n">ContentItem</span> <span
class="n">ci</span><span class="p">;</span> <span class="sr">//</span><span
class="n">the</span> <span class="n">ContentItem</span>
+<span class="sr">//</span><span class="n">the</span> <span
class="n">URI</span> <span class="n">is</span> <span class="n">available</span>
<span class="n">as</span> <span class="n">constant</span> <span
class="n">of</span> <span class="n">the</span> <span
class="n">ExecutionMetadata</span> <span class="n">class</span>
+<span class="n">UriRef</span> <span class="n">contentPartURI</span> <span
class="o">=</span> <span class="n">ExecutionMetadata</span><span
class="o">.</span><span class="n">CHAIN_EXECUTION</span><span class="p">;</span>
+
+<span class="n">MGraph</span> <span class="n">executionMetadata</span> <span
class="o">=</span> <span class="n">ci</span><span class="o">.</span><span
class="n">getPart</span><span class="p">(</span><span
class="n">contentPartURI</span><span class="p">,</span><span
class="n">MGraph</span><span class="o">.</span><span
class="n">class</span><span class="p">);</span>
+</pre></div>
+
+
+<p>The ExecutionMetadata are stored as read-/writeable RDF graph. To parse an
read-only version to other components one can use the "getGraph()" method
defined by MGraph.</p>
+<h3 id="getting_details_about_the_emchainexecution">Getting details about the
em:ChainExecution</h3>
+<p>The following code segments show how to access information about the
execution of the enhancement process for a <a
href="contentitem.html">ContentItem</a>. All directly accessed Methods in the
examples below are static imports from one of the following two utility classes
part of the "org.apache.stanbol.enhancer.servicesapi" module.</p>
+<ul>
+<li>ExecutionPlanHelper: Utility class that provides methods for reading and
creating <a href="chains/executionplan.html">ExecutionPlan</a>.</li>
+<li>ExecutionMetadataHelper: Utility class for reading and manipulating the
ExecutionMetadata</li>
+<li>
+<p>EnhancementEngineHelper: Utility that contains general purpose RDF
utilities.</p>
+<p>ContentItem ci; //the ContentItem
+MGraph em; //the ExecutionMetadata</p>
+<p>//get the ChainExecution, ExecutionPlan and the name of the Chain
+NonLiteral ce = getChainExecution(em,ci.getUri());
+if(ce != null){
+ NonLiteral ep = getExecutionPlan(em,ce);
+ String chainName = getString(em,ep,ExecutionPlan.CHAIN);
+} else {
+ log.warn("ExecutionMetadata of not contain information for "
+ + "ContentItem {}!",ci.getUri());
+}</p>
+<p>//get the EngineExecutions and the name of the Engines
+Set<NonLiteral> executions = getExecutions(em,ce);
+for(NonLiteral ex : executions){
+ NonLiteral en = getExecutionNode(em,ex);
+ if(en != null){
+ String engineName = getEngine(em,en);
+ boolean optional = isOptional(em,en);
+ } else { //maybe a sub-chain execution
+ //currently not supported, but might
+ //added in future versions
+ }
+ UriRef status = getStatus(em,ex);
+ Date started = getStarted(em,ex);
+ Date completed = getCompleted(em,ex);
+}</p>
+</li>
+</ul>
</div>
<div id="footer">
