Author: buildbot
Date: Fri Nov 23 13:34:41 2012
New Revision: 839320
Log:
Staging update by buildbot for stanbol
Modified:
websites/staging/stanbol/trunk/content/ (props changed)
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpchunker.html
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlppos.html
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpsentence
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlptokenizer.html
Propchange: websites/staging/stanbol/trunk/content/
------------------------------------------------------------------------------
--- cms:source-revision (original)
+++ cms:source-revision Fri Nov 23 13:34:41 2012
@@ -1 +1 @@
-1412880
+1412885
Modified:
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpchunker.html
==============================================================================
---
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpchunker.html
(original)
+++
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpchunker.html
Fri Nov 23 13:34:41 2012
@@ -86,12 +86,12 @@
<ul> <li><a href="/">Home</a></li> <li class="item"><a
href="/docs/">Docs</a></li> <li class="item"><a
href="/docs/trunk/">Trunk</a></li> <li class="item"><a
href="/docs/trunk/components/">Components</a></li> <li class="item"><a
href="/docs/trunk/components/enhancer/">Enhancer</a></li> <li class="item"><a
href="/docs/trunk/components/enhancer/engines/">Engines</a></li> </ul>
</div>
<h1 class="title">OpenNLP Chunker Engine</h1>
- <p>The OpenNLP Chunker Engine support the detection of Phrases (Noun,
Verb, ...) within the parsed Text. For that it uses the OpenNLP Chunker
feature. Detected Phrases are added as <em>Chunk_s to the </em><a
href="../nlp/analyzedtext">AnalyzedText</a><em> content part. In addition added
_Chunk_s are annotated with an <a
href="../nlp/nlpannotations#phrase-annotations">Phrase Annotation</a> providing
the type of the Phrase represented by the _Chunk</em>.</p>
+ <p>The OpenNLP Chunker Engine support the detection of Phrases (Noun,
Verb, ...) within the parsed Text. For that it uses the OpenNLP Chunker
feature. Detected Phrases are added as <em>Chunks</em> to the <em><a
href="../nlp/analyzedtext">AnalyzedText</a></em> content part. In addition
added <em>Chunks</em> are annotated with an <a
href="../nlp/nlpannotations#phrase-annotations">Phrase Annotation</a> providing
the type of the Phrase represented by the <em>Chunk</em>.</p>
<h2 id="consumed-information">Consumed information</h2>
<ul>
<li><strong>Language</strong> (required): The language of the text needs to be
available. It is read as specified by <a
href="https://issues.apache.org/jira/browse/STANBOL-613";>STANBOL-613</a> from
the metadata of the ContentItem. Effectively this means that any Stanbol
Language Detection engine will need to be executed before the OpenNLP POS
Tagging Engine.</li>
<li><strong>Tokens with POS annotations</strong> (required): This Engine needs
the Text to be tokenized and POS tagged. Even more the POS tags need to be
compatible with the POS tags used to train the Chunker model. This effectively
means that this Engine will only work as expected if the POS tagging was done
by the OpenNLP POS Tagging Engine configured with a POS model using the same
POS tag set as used for training the chunker model.</li>
-<li><strong>Sentences</strong> (optional): In case <em>Sentence_s are
available in the _AnalyzedText</em> content part the tokenization of the text
is done sentence by sentence. Otherwise the whole text is tokenized at
once.</li>
+<li><strong>Sentences</strong> (optional): In case <em>Sentences</em> are
available in the <em>AnalyzedText</em> content part the tokenization of the
text is done sentence by sentence. Otherwise the whole text is tokenized at
once.</li>
</ul>
<h2 id="configuration">Configuration</h2>
<p>The OpenNLP Chunker Engine provides a default service instance
(configuration policy is optional) that is configured to process all languages.
For German the model parameter is set to
'OpenNLP_1.5.1-German-Chunker-TigerCorps07.zip' a chunker model that only
detects Noun Phrases. This model is included in the
'o.a.stanbol.data.opennlp.lang.de' module. This Engine instance uses the name
'opennlp-chunker' and has a service ranking of '-100'.</p>
Modified:
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlppos.html
==============================================================================
---
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlppos.html
(original)
+++
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlppos.html
Fri Nov 23 13:34:41 2012
@@ -90,7 +90,7 @@
<h2 id="consumed-information">Consumed information</h2>
<ul>
<li><strong>Language</strong> (required): The language of the text needs to be
available. It is read as specified by <a
href="https://issues.apache.org/jira/browse/STANBOL-613";>STANBOL-613</a> from
the metadata of the ContentItem. Effectively this means that any Stanbol
Language Detection engine will need to be executed before the OpenNLP POS
Tagging Engine.</li>
-<li><strong>Sentences</strong> (optional): In case <em>Sentence_s are
available in the _AnalyzedText</em> content part the tokenization of the text
is done sentence by sentence. If no _Sentence_s are available this engine
detects sentences if a sentence detection model is available for that language
(see below for more information). If no _Sentence_s are present and no OpenNLP
sentence detection model is available for the language of the processed text,
than the whole text is processed as a single sentence.</li>
+<li><strong>Sentences</strong> (optional): In case <em>Sentences</em> are
available in the <em>AnalyzedText</em> content part the tokenization of the
text is done sentence by sentence. If no <em>Sentences</em> are available this
engine detects sentences if a sentence detection model is available for that
language (see below for more information). If no <em>Sentences</em> are present
and no OpenNLP sentence detection model is available for the language of the
processed text, than the whole text is processed as a single sentence.</li>
<li><strong>Tokens</strong> (optional): Foe POS tagging the Text needs to be
tokenized. This Engine tries to consume <em>Tokens</em> from the
<em>AnalyzedText</em> content part. If no Tokens are available it uses the
OpenNLP tokenizer to tokenize the text (see below for more information).</li>
</ul>
<h2 id="pos-tagging">POS Tagging</h2>
@@ -125,14 +125,14 @@
<li>Spanish: based on the PAROLE TagSet mapping to the <a
href="http://nlp2rdf.lod2.eu/olia/";>OLiA Ontology</a> (<a
href="http://purl.org/olia/parole_es_cat.owl";>annotation model</a>)</li>
<li>Danish: mappings for the PAROLE Tagset as described by <a
href="http://korpus.dsl.dk/paroledoc_en.pdf";>this paper</a>.</li>
<li>Portuguese: mappings based on the <a
href="http://beta.visl.sdu.dk/visl/pt/symbolset-floresta.html";>PALAVRAS tag
set</a></li>
-<li>Dutch: mappings based on the WOTAN Tagset for Dutch as described by
<em>"WOTAN: Een automatische grammatikale tagger voor het Nederlands", doctoral
dissertation, Department of language & Speech, Nijmegen University (renamed
to Radboud University), december 1994."</em>. <em>NOTE</em> that this TagSet
does NOT distinguish between _ProperNoun_s and _CommonNoun_s.</li>
+<li>Dutch: mappings based on the WOTAN Tagset for Dutch as described by
<em>"WOTAN: Een automatische grammatikale tagger voor het Nederlands", doctoral
dissertation, Department of language & Speech, Nijmegen University (renamed
to Radboud University), december 1994."</em>. <em>NOTE</em> that this TagSet
does NOT distinguish between <em>ProperNouns</em> and _CommonNoun_s.</li>
<li>Swedish: based on the <a
href="http://w3.msi.vxu.se/users/nivre/research/MAMBAlex.html";>Lexical
categories in MAMBA</a></li>
</ul>
<p><strong>TODO:</strong> Currently the Engine is limited to those TagSets as
it is not yet possible to extend this by additional one.</p>
<h2 id="tokenizing-and-sentence-detection-support">Tokenizing and Sentence
Detection Support</h2>
-<p>The OpenNLP POS Tagging engine implicitly supports tokenizing and sentence
detection. That means if the <em><a
href="../nlp/analysedtext">AnalyzedText</a></em> is not present or does not
contain _Token_s than this engine will use the OpenNLP Tokenizer to tokenize
the text. If no language specific OpenNLP tokenizer model is available, than it
will use the SIMPLE_TOKENIZER.</p>
-<p>Sentence detection is only done if no <em>Sentence_s are present in the
_AnalyzedText</em> AND if a language specific sentence detection model is
available.</p>
-<p><strong>NOTE</strong>: Support for Tokenizing and Sentence Detection is not
a replacement for explicitly adding a Tokenizing and Sentence Detection Engine
to a Enhancement Chain as this Engine does not guarantee that <em>Token_s or
_Sentence_s are added to the _AnalyzedText</em> content part. If no POS model
is available for a language or a language is not configured to be processed
there will be no _Token_s nor _Sentence_s added. Chains the relay on _Token_s
and/or _Sentence_s MUST explicitly include a Tokenizing and Sentence detection
engine!</p>
+<p>The OpenNLP POS Tagging engine implicitly supports tokenizing and sentence
detection. That means if the <em><a
href="../nlp/analysedtext">AnalyzedText</a></em> is not present or does not
contain <em>Tokens</em> than this engine will use the OpenNLP Tokenizer to
tokenize the text. If no language specific OpenNLP tokenizer model is
available, than it will use the SIMPLE_TOKENIZER.</p>
+<p>Sentence detection is only done if no <em>Sentences</em> are present in the
<em>AnalyzedText</em> AND if a language specific sentence detection model is
available.</p>
+<p><strong>NOTE</strong>: Support for Tokenizing and Sentence Detection is not
a replacement for explicitly adding a Tokenizing and Sentence Detection Engine
to a Enhancement Chain as this Engine does not guarantee that <em>Tokens</em>
or <em>Sentences</em> are added to the <em>AnalyzedText</em> content part. If
no POS model is available for a language or a language is not configured to be
processed there will be no <em>Tokens</em> nor <em>Sentences</em> added. Chains
the relay on <em>Tokens</em> and/or <em>Sentences</em> MUST explicitly include
a Tokenizing and Sentence detection engine!</p>
<h2 id="configuration">Configuration</h2>
<p><em>NOTE</em> that the OpenNLP POS Tagging engine provides a default
service instance (configuration policy is optional). This instance processes
all languages where default POS models are provided by the OpenNLP service.
This Engine instance uses the name 'opennlp-pos' and has a service ranking of
'-100'.</p>
<p>While this engine supports the default configuration including the
<strong>name</strong> <em>(stanbol.enhancer.engine.name)</em> and the
<strong>ranking</strong> <em>(service.ranking)</em> the engine also allows to
configure <strong>processed languages</strong>
<em>(org.apache.stanbol.enhancer.pos.languages)</em> and a parameter to specify
the name of the POS model used for a language.</p>
@@ -161,13 +161,13 @@
<p><strong>2. POS model parameter</strong></p>
<p>The OpenNLP POS annotation engine supports the 'model' parameter to
explicitly parse the name of the POS model used for a language. POS models are
loaded via the Stanbol DataFile provider infrastructure. That means that models
can be loaded from the {stanbol-working-dir}/stanbol/datafiles folder.</p>
<p>The syntax for parameters is as follows</p>
-<div class="codehilite"><pre><span class="p">{</span><span
class="n">language</span><span class="p">};{</span><span
class="n">param</span><span class="o">-</span><span class="n">name</span><span
class="p">}</span><span class="o">=</span><span class="p">{</span><span
class="n">param</span><span class="o">-</span><span class="n">value</span><span
class="p">}</span>
+<div class="codehilite"><pre>{language};{param-name}={param-value}
</pre></div>
<p>So to use the "my-de-pos-model.zip" for POS tagging German texts one can
use a configuration like follows</p>
-<div class="codehilite"><pre><span class="n">de</span><span
class="p">;</span><span class="n">model</span><span class="o">=</span><span
class="k">my</span><span class="o">-</span><span class="n">de</span><span
class="o">-</span><span class="nb">pos</span><span class="o">-</span><span
class="n">model</span><span class="o">.</span><span class="n">zip</span>
-<span class="o">*</span>
+<div class="codehilite"><pre>de;model=my-de-pos-model.zip
+*
</pre></div>
Modified:
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpsentence
==============================================================================
---
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpsentence
(original)
+++
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlpsentence
Fri Nov 23 13:34:41 2012
@@ -1,6 +1,6 @@
title: OpenNLP Sentence Detection Engine
-The OpenNLP Sentence Detection Engine adds _Sentence_s to the
_[AnalyzedText](../nlp/analyzedtext)_ content part. If the _AnalyzedText_
content part is not yet present it is created by this engine.
+The OpenNLP Sentence Detection Engine adds _Sentences_ to the
_[AnalyzedText](../nlp/analyzedtext)_ content part. If the _AnalyzedText_
content part is not yet present it is created by this engine.
## Consumed information
@@ -8,7 +8,7 @@ The OpenNLP Sentence Detection Engine ad
## Configuration
-The OpenNLP Sentence Detector Engine provides a default service instance
(configuration policy is optional). This instance processes all languages and
adds _Sentence_s for all languages where a OpenNLP sentence detection model is
available. This Engine instance uses the name 'opennlp-sentence' and has a
service ranking of '-100'.
+The OpenNLP Sentence Detector Engine provides a default service instance
(configuration policy is optional). This instance processes all languages and
adds _Sentences_ for all languages where a OpenNLP sentence detection model is
available. This Engine instance uses the name 'opennlp-sentence' and has a
service ranking of '-100'.
This engine supports the default configuration for Enhancement Engines
including the __name__ _(stanbol.enhancer.engine.name)_ and the __ranking__
_(service.ranking)_ In addition it is possible to configure the __processed
languages__ _(org.apache.stanbol.enhancer.sentence.languages)_ and an parameter
to specify the name of the sentence detection model used for a language.
@@ -41,10 +41,12 @@ The OpenNLP Sentence Detection engine su
The syntax for parameters is as follows
+ :::text
{language};{param-name}={param-value}
So to use the "my-de-sentence-model.zip" for detecting sentences in German
texts one can use a configuration like follows
+ :::text
de;model=my-de-sentence-model.zip
*
Modified:
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlptokenizer.html
==============================================================================
---
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlptokenizer.html
(original)
+++
websites/staging/stanbol/trunk/content/docs/trunk/components/enhancer/engines/opennlptokenizer.html
Fri Nov 23 13:34:41 2012
@@ -86,11 +86,11 @@
<ul> <li><a href="/">Home</a></li> <li class="item"><a
href="/docs/">Docs</a></li> <li class="item"><a
href="/docs/trunk/">Trunk</a></li> <li class="item"><a
href="/docs/trunk/components/">Components</a></li> <li class="item"><a
href="/docs/trunk/components/enhancer/">Enhancer</a></li> <li class="item"><a
href="/docs/trunk/components/enhancer/engines/">Engines</a></li> </ul>
</div>
<h1 class="title">OpenNLP Tokenizer Engine</h1>
- <p>The OpenNLP Tokenizer Engine adds <em>Token_s to the _AnalyzedText</em>
content part. If this content part is not yet present it adds it to the
ContentItem.</p>
+ <p>The OpenNLP Tokenizer Engine adds <em>Tokens</em> to the
<em>AnalyzedText</em> content part. If this content part is not yet present it
adds it to the ContentItem.</p>
<h2 id="consumed-information">Consumed information</h2>
<ul>
<li><strong>Language</strong> (required): The language of the text needs to be
available. It is read as specified by <a
href="https://issues.apache.org/jira/browse/STANBOL-613";>STANBOL-613</a> from
the metadata of the ContentItem. Effectively this means that any Stanbol
Language Detection engine will need to be executed before the OpenNLP POS
Tagging Engine.</li>
-<li><strong>Sentences</strong> (optional): In case <em>Sentence_s are
available in the _AnalyzedText</em> content part the tokenization of the text
is done sentence by sentence. Otherwise the whole text is tokenized at
once.</li>
+<li><strong>Sentences</strong> (optional): In case <em>Sentences</em> are
available in the <em>AnalyzedText</em> content part the tokenization of the
text is done sentence by sentence. Otherwise the whole text is tokenized at
once.</li>
</ul>
<h2 id="configuration">Configuration</h2>
<p>The OpenNLP Tokenizer engine provides a default service instance
(configuration policy is optional). This instance processes all languages.
Language specific tokenizer models are used if available. For other languages
the OpenNLP SimpleTokenizer is used. This Engine instance uses the name
'opennlp-token' and has a service ranking of '-100'.</p>
@@ -119,19 +119,19 @@
<p><strong>2. Tokenizer model parameter</strong></p>
<p>The OpenNLP Tokenizer engine supports the 'model' parameter to explicitly
parse the name of the Tokenizer model used for an language. Tokenizer models
are loaded via the Stanbol DataFile provider infrastructure. That means that
models can be loaded from the {stanbol-working-dir}/stanbol/datafiles
folder.</p>
<p>The syntax for parameters is as follows</p>
-<div class="codehilite"><pre><span class="p">{</span><span
class="n">language</span><span class="p">};{</span><span
class="n">param</span><span class="o">-</span><span class="n">name</span><span
class="p">}</span><span class="o">=</span><span class="p">{</span><span
class="n">param</span><span class="o">-</span><span class="n">value</span><span
class="p">}</span>
+<div class="codehilite"><pre>{language};{param-name}={param-value}
</pre></div>
<p>So to use the "my-de-tokenizer-model.zip" for tokenizing German texts one
can use a configuration like follows</p>
-<div class="codehilite"><pre><span class="n">de</span><span
class="p">;</span><span class="n">model</span><span class="o">=</span><span
class="k">my</span><span class="o">-</span><span class="n">de</span><span
class="o">-</span><span class="n">tokenizer</span><span class="o">-</span><span
class="n">model</span><span class="o">.</span><span class="n">zip</span>
-<span class="o">*</span>
+<div class="codehilite"><pre>de;model=my-de-tokenizer-model.zip
+*
</pre></div>
<p>To configure that the SimpleTokenizer should be used for a given language
the 'model' parameter needs to be set to 'SIMPLE' as shown in the following
example</p>
-<div class="codehilite"><pre><span class="n">de</span><span
class="p">;</span><span class="n">model</span><span class="o">=</span><span
class="n">SIMPLE</span>
-<span class="o">*</span>
+<div class="codehilite"><pre>de;model=SIMPLE
+*
</pre></div>
