Author: rwesten
Date: Thu Apr 18 12:20:02 2013
New Revision: 1469293
URL: http://svn.apache.org/r1469293
Log:
updated custom vocabulary useage scenario, fixed broken links to usage
scenarios (STANBOL-972)
Added:
stanbol/site/trunk/content/docs/trunk/enhancementworkflow.png (with props)
Modified:
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/keywordlinkingengine.mdtext
stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementstructure.mdtext
stanbol/site/trunk/content/docs/trunk/customvocabulary.mdtext
Modified:
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext
URL:
http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext?rev=1469293&r1=1469292&r2=1469293&view=diff
==============================================================================
---
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext
(original)
+++
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/entitylinking.mdtext
Thu Apr 18 12:20:02 2013
@@ -216,6 +216,7 @@ The parameters below are used to configu
* __Min Text Score__ _(enhancer.engines.linking.minTextScore)_ [0..1]::double:
The "Text Score" [0..1] represents how well the Label of an Entity matches to
the selected Span in the Text. It compares the number of matched {@link Token}
from the label with the number of Tokens enclosed by the Span in the Text an
Entity is suggested for. Not exact matches for Tokens, or if the Tokens within
the label do appear in an other order than in the text do also reduce this
score. Entities are only considered if at least one of their labels cores
higher than the minimum for all tree of _Min Labe Score_, _Min Text Match
Score_ and _Min Match Score_.
* __Min Match Score__ _(enhancer.engines.linking.minMatchScore)_
[0..1]::double: Defined as the product of the "Text Score" with the "Label
Score" - meaning that this value represents both how well the label matches the
text and how much of the label is matched with the text. Entities are only
considered if at least one of their labels cores higher than the minimum for
all tree of _Min Labe Score_, _Min Text Match Score_ and _Min Match Score_.
+* __Use EntityRankings__ _(enhancer.engines.linking.useEntityRankings)_
::boolean (default=true): Entity Rankings can be used to define the ranking
(popularity, importance, connectivity, ...) of an entity relative to other
within the knowledge base. While fise:confidence values calculated by the
EntityLinkingEngie do only represent how well a label of the entity do match
with the given section in the processed text it does make sense for manny use
cases to sort Entities with the same score based on their entity rankings (e.g.
users would expect to get "Paris (France)" suggested before "Paris (Texas)" for
Paris appearing in a text. Enabling this feature will slightly (< 0.1)
change the score of suggestions to ensure such a ordering.
#### Type Mappings Syntax
Modified:
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/keywordlinkingengine.mdtext
URL:
http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/keywordlinkingengine.mdtext?rev=1469293&r1=1469292&r2=1469293&view=diff
==============================================================================
---
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/keywordlinkingengine.mdtext
(original)
+++
stanbol/site/trunk/content/docs/trunk/components/enhancer/engines/keywordlinkingengine.mdtext
Thu Apr 18 12:20:02 2013
@@ -22,7 +22,7 @@ The example in the scene shows an config
* __Name__ _(stanbol.enhancer.engine.name)_: The name of the Enhancement
Engine. This name is used to refer an [EnhancementEngine](index.html) in
[EnhancementChain](enhancementchain.html)s
* __Referenced Site__
_(org.apache.stanbol.enhancer.engines.keywordextraction.referencedSiteId)_: The
name of the ReferencedSite of the Stanbol Entityhub that holds the controlled
vocabulary to be used for extracting Entities. "entityhub" or "local" can be
used to extract Entities managed directly by the Entityhub.
-* __Label Field__
_(org.apache.stanbol.enhancer.engines.keywordextraction.nameField)_: The name
of the property used to lookup Entities. Only a single field is supported for
performance reasons. Users that want to use values of several fields should
collect such values by an according configuration in the mappings.txt used
during indexing. This [usage scenario](../../customvocabulary.html) provides
more information on this.
+* __Label Field__
_(org.apache.stanbol.enhancer.engines.keywordextraction.nameField)_: The name
of the property used to lookup Entities. Only a single field is supported for
performance reasons. Users that want to use values of several fields should
collect such values by an according configuration in the mappings.txt used
during indexing. This [usage scenario](../../../customvocabulary.html) provides
more information on this.
* __Case Sensitivity__
_(org.apache.stanbol.enhancer.engines.keywordextraction.caseSensitive)_: This
allows to activate/deactivate case sensitive matching. It is important to
understand that even with case sensitivity activated an Entity with the label
such as "Anaconda" will be suggested for the mention of "anaconda" in the text.
The main difference will be the confidence value of such a suggestion as with
case sensitivity activated the starting letters "A" and "a" are NOT considered
to be matching. See the second technical part for details about the matching
process. Case Sensitivity is deactivated by default. It is recommended to be
activated if controlled vocabularies contain abbreviations similar to commonly
used words e.g. CAN for Canada.
* __Type Field__
_(org.apache.stanbol.enhancer.engines.keywordextraction.typeField)_: Values of
this field are used as values of the "fise:entity-types" property of created
"[fise:EntityAnnotation](../enhancementstructure.html#fiseentityannotation)"s.
The default is "rdf:type".
* __Redirect Field__
_(org.apache.stanbol.enhancer.engines.keywordextraction.redirectField)_ and
__Redirect Mode__
_(org.apache.stanbol.enhancer.engines.keywordextraction.redirectMode)_:
Redirects allow to tell the KeywordLinkingEngine to follow a specific property
in the knowledge base for matched entities. This feature e.g. allows to follow
redirects from "USA" to "United States" as defined in Wikipedia. See
"Processing of Entity Suggestions" for details. Possible valued for the
Redirect Mode are "IGNORE" - deactivates this feature; "ADD_VALUES" - uses
label, type informations of redirected entities, but keeps the URI of the
extracted entity; "FOLLOW" - follows the redirect
Modified:
stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementstructure.mdtext
URL:
http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementstructure.mdtext?rev=1469293&r1=1469292&r2=1469293&view=diff
==============================================================================
---
stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementstructure.mdtext
(original)
+++
stanbol/site/trunk/content/docs/trunk/components/enhancer/enhancementstructure.mdtext
Thu Apr 18 12:20:02 2013
@@ -11,7 +11,7 @@ Its two main purposes are to facilitate
* group Entity suggestion based on detected "Named Entities"
(disambiguation support)
* show the occurrence of detected Entities within the analyzed text
(similar to spell checker UIs)
-While this document focuses on the first Engine and provides details on how
the Stanbol Enhancement Structure it the integral part of the Stanbol Enhancer
there is also a [Usage Scenario](../enhancementusage.html) available that
focuses on how the Enhancements can be consumed by Stanbol Enhancer users.
+While this document focuses on the first Engine and provides details on how
the Stanbol Enhancement Structure it the integral part of the Stanbol Enhancer
there is also a [Usage Scenario](../../enhancementusage.html) available that
focuses on how the Enhancements can be consumed by Stanbol Enhancer users.
## Overview on the Stanbol Enhancement Structure
Modified: stanbol/site/trunk/content/docs/trunk/customvocabulary.mdtext
URL:
http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/customvocabulary.mdtext?rev=1469293&r1=1469292&r2=1469293&view=diff
==============================================================================
--- stanbol/site/trunk/content/docs/trunk/customvocabulary.mdtext (original)
+++ stanbol/site/trunk/content/docs/trunk/customvocabulary.mdtext Thu Apr 18
12:20:02 2013
@@ -5,27 +5,62 @@ The ability to work with custom vocabula
The aim of this usage scenario is to provide Apache Stanbol users with all the
required knowledge to customize Apache Stanbol to be used in their specific
domain. This includes
-* Index custom Vocabularies using the Entityhub Indexing Tool
+* Two possibilities to manage custom Vocabularies
+ 1. via the RESTful interface provided by a Managed Site or
+ 2. by using a ReferencedSite with a full local index
+* Building full local indexes with the Entityhub Indexing Tool
* Importing Indexes to Apache Stanbol
* Configuring the Stanbol Enhancer to make use of the indexed and imported
Vocabularies
## Overview
-For text enhancement and linking to external sources, the Entityhub component
of Apache Stanbol allows to work with local indexes of datasets. This has
several advantages.
+The following figure shows the typical Enhancement workflow that may start
with some preprocessing steps (e.g. the conversion of rich text formats to
plain text) followed by the Natural Language Processing phase. Next 'Semantic
Lifting' aims to connect the results of text processing and link it to the
application domain of the user. During Postprocessing those results may get
further refined.
+<p style="text-align: center;">![Typical Enhancement
Workflow](enhancementworkflow.png "The typical Enhancement Chain includes the
-- You do not rely on internet connectivity, thus it is possible to operate
offline with a huge set of entities.
-- You can do local updates of these datasets.
-- You can work with local resources, such as your LDAP directory or a specific
and private enterprise vocabulary of a specific domain.
+This usage scenario is all about the Semantic Lifting phase. This phase is
most central to for how well enhancement results to match the requirements of
the users application domain. Users that need to process health related
documents will need to provide vocabularies containing life science related
entities otherwise the Stanbol Enhancer will not perform as expected on those
documents. Similar processing Customer requests can only work if Stanbol has
access to data managed by the CRM.
-Creating your own indexes is the preferred way of working with custom
vocabularies. Small vocabularies can also be uploaded to the Entityhub as
ontologies, directly. A downside to this approach is that only one ontology per
installation is supported.
+This scenario aims to provide Stanbol users with all information necessary to
use Apache Stanbol in scenarios where domain specific vocabularies are
required.
-If you want to use multiple datasets in parallel, you have to create a local
index for these datasets and configure the Entityhub to use them. In the
following we will focuses on the main case, which is: Creating and using a
local [Apache Solr](http://lucene.apache.org/solr/) index of a custom
vocabulary, e.g. a SKOS thesaurus or taxonomy of your domain.
+## Managing Custom Vocabularies with the Stanbol Entityhub
-## Creating and working with custom local indexes
+By default the Stanbol Enhancer does use the Entityhub component for linking
Entities with mentions in the processed text. While Users may extend the
Enhancer to allow the usage of other sources this is outside of the scope of
this scenario.
-Apache Stanbol provides the machinery to start with vocabularies in standard
languages such as [SKOS](http://www.w3.org/2004/02/skos/) or
[RDF](http://www.w3.org/TR/rdf-primer/) encoded data sets. The Apache Stanbol
components, which are needed for this functionality are the Entityhub and its
indexing tool for creating and managing the index and [enhancement
engines](components/enhancer/engines) that make use of the indexes during the
enhancement process.
+The Stanbol Entityhub provides two possibilities to manage vocabularies
-To create and import your own vocabulary to the Apache Stanbol Entityhub you
need to follow the following Steps
+1. __[Managed Sites](components/entityhub/managedsite)__: A fully read/write
able storage for Entities. Once created users can use a RESTful interface to
create, update, retrieve, query and delete entities.
+2. __Referenced Site__: A read-only version of a Site that can either be used
as a local cache of remotely managed data (such as a [Linked
Data](http://linkeddata.org/) server) or use a fully local index of the
knowledge base - the relevant case in the context of this scenario.
+
+As a rule of thump users should prefer to use a __Managed Site__ if the
vocabulary does change regularly and those changes need to be reflected in
enhancement results of processed documents. A __Referenced Site__ is typically
the better choice for vocabularies that do not change on a regular base and/or
for users that what to use apply advanced rules while indexing a dataset.
+
+### Using a Entityhub Managed Site
+
+How to use a Managed Site is already described in detail by the [Documentation
of Managed Sites](components/entityhub/managedsite). To configure a new Managed
Site on the Entityhub users need to create two components:
+
+1. the _Yard_ - the storage component of the Stanbol Entityhub. While there
are multiple Yard implementations, when used for EntiyLinking the [SolrYard
implementation](components/entityhub/managedsite#configuration-of-a-solryard)
should be used. Second the
+2. the
_[YardSite](components/entityhub/managedsite#configuration-of-the-yardsite)_ -
the component that implements the ManagedSite interface.
+
+After completing those two steps an empty Managed site should be ready to use
available under
+
+ http://{stanbol-host}/entityhub/sites/{managed-site-name}/
+
+and users can start to upload the Entities of the controlled vocabulary by
using the RESTful interface such as
+
+ curl -i -X PUT -H "Content-Type: application/rdf+xml" -T {rdf-xml-data} \
+ "http://{stanbol-host}/entityhub/site/{managed-site-name}/entity";
+
+In case you have opted to use a _Managed Site_ for managing your entities you
can now skip the next section until section 'B. Configure and use the index
with the Apache Stanbol Enhancer'
+
+### Using a Entityhub Referenced Site
+
+Referenced Sites are used by the Stanbol Entityhub to reference external
knowledge bases. This can be done by configuring remote services for
dereferencing and querying information, but also by providing a full local
index of the referenced knowledge base.
+
+When using a Referenced Site in combination with the Stanbol Enhancer it is
highly recommended for performance considerations to provide a full local
index. To create such local indexes Stanbol provides the _Entityhub Indexing
Tool_. See the following section for detailed information on how to use this
tool.
+
+## Building full local indexes with the Entityhub Indexing Tool
+
+The Entityhub Indexing Tool allows to create full local indexes of knowledge
bases that can be loaded to the Stanbol Entityhub as Referenced Sites. Users
that do use Managed Sites may want to skip this section.
+
+Users of the Entityhub Indexing Tool will typically need to complete the steps
described in the following sub sections.
### Step 1 : Compile and assemble the indexing tool
@@ -66,7 +101,9 @@ This will create/initialize the default
After the initialization you will need to provide the following configurations
in files located in the configuration folder
(<code>{indexing-working-dir}/indexing/config</code>)
* Within the <code>indexing.properties</code> file you need to set the {name}
of your index by changing the value of the "name" property. In addition you
should also provide a "description". At the end of the indexing.properties file
you can also specify the license and attribution for the data you index. The
Apache Entityhub will ensure that those information will be included with any
entity data returned for requests.
-* If the data you index do use some none common namespaces you will need to
add those to the <code>mapping.txt</code> file (here is an
[example](examples/anl-mappings.txt) including default and specific mappings
for one dataset)
+* Optionally, if your data do use namespaces that are not present in
[prefix.cc](http://prefix.cc) (or the server used for indexing does not have
internet connectivity) you can manually define required prefixes by
creating/using the a <code>indexing/config/namespaceprefix.mappings</code>
file. The syntax is '<code>'{prefix}\t{namespace}\n</code>' where
'<code>{prefix} ... [0..9A..Za..z-_]</code>' and '<code>{namespace} ... must
end with '#' or '/' for URLs and ':' for URNs</code>'.
+* Optionally, if the data you index do use some none common namespaces you
will need to add those to the <code>mapping.txt</code> file (here is an
[example](examples/anl-mappings.txt) including default and specific mappings
for one dataset)
+* Optionally, if you want to use a custom SolrCore configuration the core
configuration needs to be copied to the
<code>indexing/config/{core-name}</code>. Default configuration - to start from
- can be downloaded from the [Stanbol
SVN](https://svn.apache.org/repos/asf/stanbol/trunk/entityhub/yard/solr/src/main/resources/solr/core/)
and extracted to the <code>indexing/config/</code> folder. If the {core-name}
is different from the 'name' configured in the <code>indexing.properties</code>
than the '<code>solrConf</code>' parameter of the
'<code>indexingDestination</code>' MUST be set to
'<code>solrConf:{core-name}</code>'. After those configurations users can make
custom adaptations to the SolrCore configuration used for indexing.
Finally you will also need to copy your source files into the source directory
<code>{indexing-working-dir}/indexing/resources/rdfdata</code>. All files
within this directory will be indexed. THe indexing tool support most common
RDF serialization. You can also directly index compressed RDF files.
@@ -78,23 +115,23 @@ Once all source files are in place, you
$ cd {indexing-working-dir}
$ java -Xmx1024m -jar
org.apache.stanbol.entityhub.indexing.genericrdf-*-jar-with-dependencies.jar
index
-Depending on your hardware and on complexity and size of your sources, it may
take several hours to built the index. As a result, you will get an archive of
an [Apache Solr](http://lucene.apache.org/solr/) index together with an OSGI
bundle to work with the index in Stanbol.
+Depending on your hardware and on complexity and size of your sources, it may
take several hours to built the index. As a result, you will get an archive of
an [Apache Solr](http://lucene.apache.org/solr/) index together with an OSGI
bundle to work with the index in Stanbol. Both files will be located within the
<code>indexing/dist</code> folder.
_IMPORTANT NOTES:_
* The import of the RDF files to the Jena TDB triple store - used as source
for the indexing - takes a lot of time. Because of that imported data are
reused for multiple runs of the indexing tool. This has two important effects
users need to be aware of:
- 1. Already imported RDF files should be removed from the
<code>{indexing-working-dir}/indexing/resources/rdfdata</code> to avoid to
re-import them on every run of the tool
+ 1. Already imported RDF files should be removed from the
<code>{indexing-working-dir}/indexing/resources/rdfdata</code> to avoid to
re-import them on every run of the tool. NOTE: newer versions of the Entityhub
indexing tool might automatically move successfully imported RDF files to a
different folder.
2. If the RDF data change you will need to delete the Jena TDB store so
that those changes are reflected in the created index. To do this delete the
<code>{indexing-working-dir}/indexing/resources/tdb</code> folder
* Also the destination folder
<code>{indexing-working-dir}/indexing/destination</code> is NOT deleted between
multiple calls to index. This has the effect that Entities indexed by previous
indexing calls are not deleted. While this allows to index a dataset in
multiple steps - or even to combine data of multiple datasets in a single index
- this also means that you will need to delete the destination folder if the
RDF data you index have changed - especially if some Entities where deleted.
-###Step 3 : Initialize the index within Apache Stanbol
+### Step 3 : Initialize the index within Apache Stanbol
We assume that you already have a running Apache Stanbol instance at
http://{stanbol-host} and that {stanbol-working-dir} is the working directory
of that instance on the local hard disk. To install the created index you need
to
-* copy the "{name}.solrindex.zip" file to the
<code>{stanbol-working-dir}/stanbol/datafiles</code> directory (NOTE if you run
the 0.9.0-incubating version the path is
<code>{stanbol-working-dir}/sling/datafiles</code>.
+* copy the "{name}.solrindex.zip" file to the
<code>{stanbol-working-dir}/stanbol/datafiles</code> directory (NOTE if you run
the 0.9.0-incubating version the path is
<code>{stanbol-working-dir}/sling/datafiles</code>).
* install the <code>org.apache.stanbol.data.site.{name}-{version}.jar</code>
to the OSGI environment of your Stanbol instance e.g. by using the Bundle tab
of the Apache Felix web console at
</code>http://{stanbol-host}/system/console/bundles</code>
You find both files in the <code>{indexing-working-dir}/indexing/dist/</code>
folder.
@@ -105,7 +142,8 @@ After the installation your data will be
You can use the Web UI of the Stanbol Enhancer to explore your vocabulary.
Note, that in case of big vocabulary it might take some time until the site
becomes functional.
-## B. Configure and use the index with the Apache Stanbol Enhancer
+
+## Configuring the Stanbol Enhancer for your custom Vocabularies
This section covers how to configure the Apache Stanbol Enhancer to recognize
and link entities of your custom vocabulary with processed documents.
@@ -125,54 +163,59 @@ Depending on if you want to use named en
In case named entity linking is used the linking with the custom vocabulary is
done by the [Named Entity Tagging
Engine](components/enhancer/engines/namedentitytaggingengine.html).
For the configuration of this engine you need to provide the following
parameters
-1. The "name" of the enhancement engine. It is recommended to use
"{name}Linking" - where {name} is the name of your vocabulary as used in part
A. of this scenario.
+1. The "name" of the enhancement engine. It is recommended to use
"{name}Linking" - where {name} is the name of the Entityhub Site (ReferenceSite
or ManagedSite).
2. The name of the referenced site holding your vocabulary. Here you have to
configure the {name}.
3. Enable/disable persons, organizations and places and if enabled configure
the <code>rdf:type</code> used by your vocabulary for those type. If you do not
want to restrict the type, you can also leave the type field empty.
4. Define the property used to match against the named entities detected by
the used NER engine(s).
For more detailed information please see the documentation of the [Named
Entity Tagging
Engine](components/enhancer/engines/namedentitytaggingengine.html).
-Note, that for using named entity linking you need also ensure that an
enhancement engine that provides NER is available in the [enhancement
chain](components/enhancer/chains). By default Apache Stanbol includes three
different engines that provide this feature: (1) [Named Entity Extraction
Enhancement
Engine](components/enhancer/engines/namedentityextractionengine.html) based on
[OpenNLP](http://opennlp.apache.org), (2) CELI NER engine based on the
[linguagrid.org](http://Linguagrid.org) service and (3) [OpenCalais Enhancement
Engine](components/enhancer/engines/opencalaisengine.html) based on
[OpenCalais](http://opencalais.com). Note that the later two options will
require to send your content to the according services that are not part of
your local Apache Stanbol instance.
+Note, that for using named entity linking you need also ensure that an
enhancement engine that provides NER (Named Entity Recoqunition) is available
in the [enhancement chain](components/enhancer/chains). See [Stanbol NLP
processing Language
Support](components/enhancer/nlp/#stanbol-enhancer-nlp-support) section for
detailed information on Languages with NER support.
-A typical [enhancement chain](components/enhancer/chains) for named entity
linking with your custom vocabulary might look like
+The following Example shows a [enhancement chain](components/enhancer/chains)
for named entity linking based on OpenNLP and CELI as NLP processing modules
-* "langid" - [Language Identification
Engine](components/enhancer/engines/langidengine.html) - to detect the language
of the parsed content - a pre-requirement of all NER engines
-* "ner" - for NER support in English, Spanish and Dutch via the [Named Entity
Extraction Enhancement
Engine](components/enhancer/engines/namedentityextractionengine.html)
+* "langdetect" - [Language Detection
Engine](components/enhancer/engines/langdetectengine) - to detect the language
of the parsed content - a pre-requirement of all NER engines
+* "opennlp-ner" - for NER support in English, Spanish and Dutch via the [Named
Entity Extraction Enhancement
Engine](components/enhancer/engines/namedentityextractionengine.html)
* "celiNer" - for NER support in French and Italien via the CELI NER engine
* "{name}Linking - the [Named Entity Tagging
Engine](components/enhancer/engines/namedentitytaggingengine.html) for your
vocabulary as configured above.
Both the [weighted chain](components/enhancer/chains/weightedchain.html) and
the [list chain](components/enhancer/chains/listchain.html) can be used for the
configuration of such a chain.
-### Configure Keyword Linking
-
-In case you want to use keyword linking to extract and link entities of your
vocabulary you will need to configure the [Keyword Linking
Engine](components/enhancer/engines/keywordlinkingengine.html) accordingly.
+### Configuring Named Entity Linking
-Here are the most important configuration options provided by the Keyword
Linking Engine when configured via the [configuration
tab](http://localhost:8080/system/console/configMgr) of the Apache Felix web
console - http://{host}:{port}/system/console/configMgr. For the full list and
detailed information please see the
[documentation](components/enhancer/engines/keywordlinkingengine.html)).
+First it is important to note the difference between _Named Entity Linking_
and _Entity Linking_. While _Named Entity Linking_ only considers _Named
Entities_ detected by NER (Named Entity Recognition) _Entity Linking_ does work
on Words (Tokens). Because of that is has much lower NLP requirements and can
even operate for languages where only word tokenization is supported. However
extraction results AND performance do greatly improve with POS (Part of Speech)
tagging support. Also Chunking (Noun Phrase detection), NER and Lemmatization
results can be consumed by Entity Linking to further improve extraction
results. For details see the documentation of the [Entity Linking
Process](components/enhancer/engines/entitylinking#linking-process).
-1. The "Name" of the enhancement engine. It is recommended to use
"{name}Keyword" - where {name} is the name of your vocabulary as used in part
A. of this scenario
-2. The name of the "Referenced Site" holding your vocabulary. Here you have to
configure the {name}
-3. The "Label Field" is the URI of the property in your vocabulary providing
the labels used for matching. You can only use a single field. If you want to
use values of several fields you have two options: (1) to adapt your indexing
configuration to copy the values of those fields to a single one (e.g. the
values of "skos:prefLabel" and "skos:altLabel" are copied to "rdfs:label" in
the default configuration of the Entityhub indexing tool (see
{indexing-working-dir}/indexing/config/mappings.txt) (2) to configure multiple
Keyword Linking Engine(s) - one for each label field. Option (1) is preferable
as long as you do not need to use different configurations for the different
labels.
-4. The "Type Mappings" might be interesting for you if your vocabulary
contains custom types as those mappings can be used to map 'rdf:type's of
entities in your vocabulary to 'dc:type's used for 'fise:TextAnnotation's -
created by the Apache Stanbol Enhancer to annotate occurrences of extracted
entities in the parsed text. See the [type mapping
syntax](components/enhancer/engines/keywordlinkingengine.html#type-mappings-syntax)
and the [usage scenario for the Apache Stanbol Enhancement
Structure](enhancementusage.html#entity-tagging-with-disambiguation-support)
for details.
+The second big difference is that _Named Entity Linking_ can only support
Entity types supported by the NER modles (Persons, Organizations and Places).
_Entity Linking_ does not have this restriction. This advantage comes also with
the disadvantage that Entity Lookups to the Controlled Vocabulary are only
based on Label similarities. _Named Entity Linking_ does also use the type
information provided by NER.
-A typical [enhancement chain](components/enhancer/chains) for named entity
linking with your vocabulary might look like
+To use _Entity Linking_ with a custom Vocabulary Users need to configure an
instance of the [Entityhub Linking
Engine](components/enhancer/engines/entityhublinking). While this Engine
provides more than twenty configuration parameters the following list provides
an overview about the most important. For detailed information please see the
documentation of the Engine.
-* "langid" - [Language Identification
Engine](components/enhancer/engines/langidengine.html) - to detect the language
of the parsed content - a pre-requirement of the Keyword Linking Engine.
-* "{name}Keyword - the [Keyword Linking
Engine](components/enhancer/engines/keywordlinkingengine.html) for your
vocabulary as configured above.
+1. The "Name" of the enhancement engine. It is recommended to use something
like "{name}Extraction" - where {name} is the name of the Entityhub Site
+2. The name of the "Managed- / Referenced Site" holding your vocabulary. Here
you have to configure the {name}
+3. The "Label Field" is the URI of the property in your vocabulary providing
the labels used for matching. You can only use a single field. If you want to
use values of several fields you have two options: (1) to adapt your indexing
configuration to copy the values of those fields to a single one (e.g. the
values of "skos:prefLabel" and "skos:altLabel" are copied to "rdfs:label" in
the default configuration of the Entityhub indexing tool (see
{indexing-working-dir}/indexing/config/mappings.txt) (2) to configure multiple
EntityubLinkingEngines - one for each label field. Option (1) is preferable as
long as you do not need to use different configurations for the different
labels.
+4. The "Link ProperNouns only": If the custom Vocabulary contains Proper Nouns
(Named Entities) than this parameter should be activated. This options causes
the Entity Linking process to not making queries for commons nouns and by that
receding the number of queries agains the controlled vocabulary by ~70%.
However this is not feasible if the vocabulary does contain Entities that are
common nouns in the language.
+5. The "Type Mappings" might be interesting for you if your vocabulary
contains custom types as those mappings can be used to map 'rdf:type's of
entities in your vocabulary to 'dc:type's used for 'fise:TextAnnotation's -
created by the Apache Stanbol Enhancer to annotate occurrences of extracted
entities in the parsed text. See the [type mapping
syntax](components/enhancer/engines/keywordlinkingengine.html#type-mappings-syntax)
and the [usage scenario for the Apache Stanbol Enhancement
Structure](enhancementusage.html#entity-tagging-with-disambiguation-support)
for details.
+
+The following Example shows an Example of an [enhancement
chain](components/enhancer/chains) using OpenNLP for NLP
+
+* "langdetect" - [Language Detection
Engine](components/enhancer/engines/langdetectengine) - to detect the language
of the parsed content - a pre-requirement of all NER engines
+* opennlp-sentence - [Sentence detection with
OpenNLP](components/enhancer/engines/opennlpsentence)
+* opennlp-token - [OpenNLP based Word
tokenization](components/enhancer/engines/opennlptokenizer). Works for all
languages where white spaces can be used to tokenize.
+* opennlp-pos - [OpenNLP Part of Speech
tagging](components/enhancer/engines/opennlppos)
+* opennlp-chunker - The [OpenNLP
chunker](components/enhancer/engines/opennlpchunker) provides Noun Phrases
+* "{name}Extraction - the [Entityhub Linking
Engine](components/enhancer/engines/entityhublinking) configured for the custom
vocabulary.
Both the [weighted chain](components/enhancer/chains/weightedchain.html) and
the [list chain](components/enhancer/chains/listchain.html) can be used for the
configuration of such a chain.
+The documentation of the Stanbol NLP processing module provides [detailed
Information](components/enhancer/nlp/#stanbol-enhancer-nlp-support) about
integrated NLP frameworks and suupported languages.
+
### How to use enhancement chains
-In the default configuration the Apache Stanbol Enhancer provides two
enhancement chains:
+In the default configuration the Apache Stanbol Enhancer provides several
enhancement chains including:
-1) a "default" chain that includes all currently active [enhancement
engines](components/enhancer/engines) and
+1) a "default" chain providing _Named Entity Linking_ based on DBpedia and
_Entity Linking_ based on the Entityhub
2) the "language" chain that is intended to be used to detect the language of
parsed content.
+3) a "dbpedia-proper-noun-linking" chain showing _Named Entity Linking_ based
on DBpedia
-As soon as Apache Stanbol users start to add own vocabularies to the Apache
Stanbol Entityhub and configure [Named Entity Tagging
Engine](components/enhancer/engines/namedentitytaggingengine.html) or [Keyword
Linking Engine](components/enhancer/engines/keywordlinkingengine.html), the
default chain, which includes all active engines, may become unusable. Most
likely users want to deactivate the "default" chain and configure their own.
This section provides more information on how to do that.
-
-__Deactivate the chain of all active enhancement engines__
-
-Users that add additional enhancement engines might need to deactivate the
enhancement chain that includes all active engines. This can be done in the
configuration tab of the Apache Felix web console -
[http://{stabol-host}/system/console/configMgr](http://localhost:8080/system/console/configMgr).
Open the configuration dialog of the "Apache Stanbol Enhancer Chain: Default
Chain" component and deactivate it.
__Change the enhancement chain bound to "/enhancer"__
Added: stanbol/site/trunk/content/docs/trunk/enhancementworkflow.png
URL:
http://svn.apache.org/viewvc/stanbol/site/trunk/content/docs/trunk/enhancementworkflow.png?rev=1469293&view=auto
==============================================================================
Binary file - no diff available.
Propchange: stanbol/site/trunk/content/docs/trunk/enhancementworkflow.png
------------------------------------------------------------------------------
svn:mime-type = application/octet-stream
