This is an automated email from the ASF dual-hosted git repository.
sergeykamov pushed a commit to branch NLPCRAFT-513
in repository https://gitbox.apache.org/repos/asf/incubator-nlpcraft-website.git
The following commit(s) were added to refs/heads/NLPCRAFT-513 by this push:
new be72dc4 WIP.
be72dc4 is described below
commit be72dc48fd6be347dda252c1baa93e3b8b3f981f
Author: skhdl <[email protected]>
AuthorDate: Mon Oct 24 16:33:54 2022 +0400
WIP.
---
_includes/left-side-menu.html | 33 +--
api-review.html | 29 +-
built-components.html | 50 +---
custom-components.html | 174 +++++++++++-
examples/calculator.html | 1 -
installation.html | 110 +++-----
semantic.html | 627 ++++++++++++++++++++++++++++++++++++++++++
7 files changed, 875 insertions(+), 149 deletions(-)
diff --git a/_includes/left-side-menu.html b/_includes/left-side-menu.html
index 93907e6..40eec54 100644
--- a/_includes/left-side-menu.html
+++ b/_includes/left-side-menu.html
@@ -39,31 +39,17 @@
{% endif %}
</li>
<li>
- {% if page.id == "built-components" %}
- <a class="active" href="/custom-components.html">Custom components</a>
- {% else %}
- <a href="/custom-components.html">Custom components</a>
- {% endif %}
- </li>
- <li>
- {% if page.id == "installation" %}
- <a class="active" href="/installation.html">Installation</a>
- {% else %}
- <a href="/installation.html">Installation</a>
- {% endif %}
- </li>
- <li>
- {% if page.id == "first_example" %}
- <a class="active" href="/first-example.html">First Example</a>
+ {% if page.id == "semantic" %}
+ <a class="active" href="/semantic.html">Semantic model</a>
{% else %}
- <a href="/first-example.html">First Example</a>
+ <a href="/semantic.html">Semantic model</a>
{% endif %}
</li>
<li>
- {% if page.id == "data_model" %}
- <a class="active" href="/data-model.html">Data Model</a>
+ {% if page.id == "built-components" %}
+ <a class="active" href="/custom-components.html">Custom components</a>
{% else %}
- <a href="/data-model.html">Data Model</a>
+ <a href="/custom-components.html">Custom components</a>
{% endif %}
</li>
<li>
@@ -80,6 +66,13 @@
<a href="/short-term-memory.html">Short-Term Memory</a>
{% endif %}
</li>
+ <li>
+ {% if page.id == "installation" %}
+ <a class="active" href="/installation.html">Installation</a>
+ {% else %}
+ <a href="/installation.html">Installation</a>
+ {% endif %}
+ </li>
<li class="side-nav-title">Examples</li>
<li>
{% if page.id == "calculator" %}
diff --git a/api-review.html b/api-review.html
index 2138b8f..82407ad 100644
--- a/api-review.html
+++ b/api-review.html
@@ -76,29 +76,29 @@ id: overview
</ul>
<p>
- Base client methods:
+ <b>Base client methods:</b>
</p>
<ul>
<li>
- <code>ask</code> passes user text input to the model and
receives back triggered callback method execution result or
+ <code>ask()</code> passes user text input to the model and
receives back triggered callback method execution result or
rejection exception if there isn't any triggered intents.
</li>
<li>
- <code>debugAsk</code> passes user text input to the model and
receives back callback and its parameters or
+ <code>debugAsk()</code> passes user text input to the model
and receives back callback and its parameters or
rejection exception if there isn't any triggered intents.
Main difference from <code>ask</code> that triggered intent
callback method is not called.
This method and this parameter can be useful for tests
scenarios.
</li>
<li>
- <code>clearStm</code> clears STM state. Memory is cleared
wholly or with some predicate.
+ <code>clearStm()</code> clears STM state. Memory is cleared
wholly or with some predicate.
Loot at <a href="short-term-memory.html">Conversation</a>
chapter to get more details.
</li>
<li>
- <code>clearDialog</code> clears dialog state. Dialog is
cleared wholly or with some predicate.
+ <code>clearDialog()</code> clears dialog state. Dialog is
cleared wholly or with some predicate.
Loot at <a href="short-term-memory.html">Conversation</a>
chapter to get more details.
</li>
<li>
- <code>close</code> - Closes client. You can't call another
client's methods after this method was closed.
+ <code>close()</code> - Closes client. You can't call another
client's methods after this method was closed.
</li>
</ul>
</section>
@@ -142,20 +142,29 @@ id: overview
<code>Token</code> represented as <a
href="apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a>.
It is simple string, part of user input, which split according
to some rules, for instance by spaces and some additional conditions, which
depends on language and some expectations.
So user input "<b>Where is it?</b>" contains four tokens:
"<b>Where</b>", "<b>is</b>", "<b>it</b>", "<b>?</b>".
+ Tokens data are input for searching <code>entities</code>.
</li>
<li>
<code>Entity</code> represented as <a
href="apis/latest/org/apache/nlpcraft/NCEntity.html">NCEntity</a>.
- According to wikipedia, named entity is a real-world object,
such as a person, location, organization, product, etc., that can be denoted
with a proper name. It can be abstract or have a physical existence. Each
entity can contain one or more tokens.
+ According to wikipedia, named entity is a real-world object,
such as a person, location, organization,
+ product, etc., that can be denoted with a proper name. It can
be abstract or have a physical existence.
+ Each entity can contain one or more tokens.
+ Entities data are input for searching <a
href="intent-matching.html">Intent matching</a> conditions.
</li>
<li>
<code>Variant</code> represented as <a
href="apis/latest/org/apache/nlpcraft/NCVariant.html">NCVariant</a>.
- List of entities. Potentially, each token can be recognized as
different entities, so user input can be processed as set of variants. For
example user input "Mercedes" can be processed as 2 variants, both of them
contains single element list of entities: car brand or Spanish family name.
+ List of entities. Potentially, each token can be recognized as
different entities,
+ so user input can be processed as set of variants.
+ For example user input "Mercedes" can be processed as 2
variants,
+ both of them contains single element list of entities: car
brand or Spanish family name.
+ When words are not overlapped with different
<code>entities</code> there is only one
+ <code>variant</code> detected.
</li>
</ul>
<p>
- Back to <code>Pipeline</code> represented as <a
href="apis/latest/org/apache/nlpcraft/NCPipeline.html">pipeline</a>.
- Pipeline should be created based in following components:
+ Back to <code>pipeline</code> represented as <a
href="apis/latest/org/apache/nlpcraft/NCPipeline.html">NCPipeline</a>.
+ <code>Pipeline</code> should be created based in following
components:
</p>
<ul>
<li>
diff --git a/built-components.html b/built-components.html
index 055acb9..f162a8e 100644
--- a/built-components.html
+++ b/built-components.html
@@ -108,7 +108,7 @@ id: overview
There are provided a number of built implementations, all of
them for English language.
<ul>
<li>
- <a
href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
converts NLP tokens into entities with four mandatory properties:
+ <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCNLPEntityParser.html">NCNLPEntityParser</a>
converts NLP tokens into entities with four mandatory properties:
<code>nlp:token:text</code>,
<code>nlp:token:index</code>, <code>nlp:token:startCharIndex</code> and
<code>nlp:token:endCharIndex</code>. However, if any
other properties were added into
processed tokens by <code>NCTokenEnricher</code>
components, they will be copied also with names
@@ -119,7 +119,7 @@ id: overview
<a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCOpenNLPTokenParser.html">NCOpenNLPTokenParser</a>
is wrapper on <a href="https://opennlp.apache.org/";>Apache OpenNLP</a> NER
components.
Look at the supported <b>Name Finder</b> models <a
href="https://opennlp.sourceforge.net/models-1.5/";>here</a>.
For example for English language are accessible:
<code>Location</code>, <code>Money</code>,
- <code>Person</code>, <code>Organizationon</code>,
<code>Date</code>, <code>Time</code> and <code>Percentage</code>.
+ <code>Person</code>, <code>Organization</code>,
<code>Date</code>, <code>Time</code> and <code>Percentage</code>.
</li>
<li>
<code>NCStanfordNLPEntityParser</code> is wrapper on
<a href="https://nlp.stanford.edu/";>Stanford NLP</a> NER components.
@@ -127,7 +127,8 @@ id: overview
</li>
<li>
<a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a>
is entity parser which is based on list of synonyms elements.
- This component is described with more details below in
<a href="#semantic">Semantic enrichers</a> section.
+ It is very important component which allow to solve a
wide range of tasks.
+ Separated chapter <a href="semantic.html">Semantic
model</a> is dedicated to its detailed description.
</li>
</ul>
</li>
@@ -162,49 +163,6 @@ id: overview
</ul>
</section>
- <section id="semantic">
- <h2 class="section-title">Semantic entity parser <a href="#"><i
class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
-
- <p>
- Semantic entity parser <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a>
- is the implementation of <a
href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
- which deserves a special mention.
- It describes list of <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
- which are represented <a
href="https://en.wikipedia.org/wiki/Named-entity_recognition";>Named
entities</a>.
- </p>
-
- <p>
- Each <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
- is presented by <code>id</code>, <code>groups</code>,
<code>synonyms</code>, <code>values</code> and <code>properties</code>.
- <p>
-
- <section id="element-synonyms">
- <h3>Element synonyms</h3>
- <p>TODO...</p>
- </section>
-
- <section id="element-values">
- <h3>Element values</h3>
- <p>TODO...</p>
- </section>
-
- <section id="element-groups">
- <h3>Element values</h3>
- <p>TODO...</p>
- </section>
-
- <section id="element-macros">
- <h3>Element macros</h3>
- <p>TODO...</p>
- </section>
-
- <section id="external-declaration">
- <h3>External JSON/YAML Declaration</h3>
- <p>TODO...</p>
- </section>
-
- </section>
-
<section id="examples">
<h2 class="section-title">Examples <a href="#"><i class="top-link fas
fa-fw fa-angle-double-up"></i></a></h2>
diff --git a/custom-components.html b/custom-components.html
index 9e0030e..529acbd 100644
--- a/custom-components.html
+++ b/custom-components.html
@@ -90,10 +90,8 @@ id: overview
You have to implement <a
href="apis/latest/org/apache/nlpcraft/NCTokenEnricher.html">NCTokenEnricher</a>
trait.
</p>
<p>
- <a href="apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a>
are used in
- <a href="intent-matching.html">Intent matching</a>. NlpCraft
provides a numeric of built token enricher
- implementations for English language.
- You may want to create your own or extends existing. Look at the
following example:
+ Tokens enricher is component which allows to add additional
properties to prepared tokens.
+ These tokens properties are used later when entities detection.
</p>
<pre class="brush: scala, highlight: [25, 26]">
import org.apache.nlpcraft.*
@@ -139,6 +137,27 @@ id: overview
<p>
You have to implement <a
href="apis/latest/org/apache/nlpcraft/NCTokenValidator.html">NCTokenValidator</a>
trait.
</p>
+
+ <p>
+ There are tokens are inspected and exception can be thrown from
user code to break user input processing.
+ </p>
+
+ <pre class="brush: scala, highlight: [3]">
+ new NCTokenValidator:
+ override def validate(req: NCRequest, cfg: NCModelConfig,
toks: List[NCToken]): Unit =
+ if toks.exists(_.contains("restrictionFlag"))
+ then throw new NCException("Sentence cannot be
processed.")
+ </pre>
+
+ <ul>
+ <li>
+ There is anonymous instance of <a
href="apis/latest/org/apache/nlpcraft/NCTokenValidator.html">NCTokenValidator</a>
+ created.
+ </li>
+ <li>
+ <code>Lines 3</code> defines the rule when exception should be
thrown and sentence processing should be stopped.
+ </li>
+ </ul>
</section>
<section id="entity-parser">
@@ -146,6 +165,37 @@ id: overview
<p>
You have to implement <a
href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
trait.
</p>
+
+ <p>
+ Most important component which finds user specific data.
+ These defined entities are input for <a
href="intent-matching.html">Intent matching</a> conditions.
+ If built <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a>
+ is not enough, you can implement your own NER searching here.
+ There is point for potential integrations with neural networks or
any other solutions which
+ help you find and mark your domain specific named entities.
+ </p>
+
+ <pre class="brush: scala, highlight: [5]">
+ import org.apache.nlpcraft.*
+
+ class CommentsEntityParser extends NCEntityParser :
+ def parse(req: NCRequest, cfg: NCModelConfig, toks:
List[NCToken]): List[NCEntity] =
+ if req.getText.trim.startsWith("--") then
+ List(
+ new NCPropertyMapAdapter with NCEntity :
+ override def getTokens: List[NCToken] = toks
+ override def getRequestId: String =
req.getRequestId
+ override def getId: String = "comment"
+ )
+ else
+ List.empty
+ </pre>
+ <ul>
+ <li>
+ In given example whole input sentence is marked as single
element <code>comment</code> if
+ condition defined on <code>line 5</code> is <code>true</code>.
+ </li>
+ </ul>
</section>
<section id="entity-enricher">
@@ -153,13 +203,85 @@ id: overview
<p>
You have to implement <a
href="apis/latest/org/apache/nlpcraft/NCEntityEnricher.html">NCEntityEnricher</a>
trait.
</p>
+ <p>
+ Entity enricher is component which allows to add additional
properties to prepared entities.
+ Can be useful for extending existing entity enrichers
functionality.
+ </p>
+
+ <pre class="brush: scala, highlight: [4, 10, 11]">
+ import org.apache.nlpcraft.*
+
+ object CityPopulationEntityEnricher:
+ val citiesPopulation: Map[String, Int] =
someExternalService.getCitiesPopulation()
+
+ import CityPopulationEntityEnricher.*
+
+ class CityPopulationEntityEnricher extends NCEntityEnricher :
+ def enrich(req: NCRequest, cfg: NCModelConfig, ents:
List[NCEntity]): Unit =
+ ents.
+ filter(_.getId == "city").
+ foreach(e => e.put("city:population",
citiesPopulation(e("city:name"))))
+ </pre>
+
+ <ul>
+ <li>
+ <code>Line 4</code> defines getting cities population data
from some external service.
+ </li>
+ <li>
+ <code>Line 10</code> filters entities by <code>ID</code>.
+ </li>
+ <li>
+ <code>Line 11</code> enriches entities by new
<code>city:population</code> property.
+ </li>
+ </ul>
</section>
<section id="entity-mapper">
- <h2 class="section-title">Entity enricher<a href="#"><i
class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+ <h2 class="section-title">Entity mapper<a href="#"><i class="top-link
fas fa-fw fa-angle-double-up"></i></a></h2>
<p>
You have to implement <a
href="apis/latest/org/apache/nlpcraft/NCEntityMapper.html">NCEntityMapper</a>
trait.
</p>
+
+ <p>
+ Entity mapper is component which allows to map one set of entities
into another after the entities
+ were parsed and enriched. Can be useful for building complex
parsers based on existing.
+ </p>
+
+ <pre class="brush: scala, highlight: [4, 10, 12, 13, 14]">
+ import org.apache.nlpcraft.*
+
+ object CityPopulationEntityMapper:
+ val citiesPopulation: Map[String, Int] =
externalService.getCitiesPopulation()
+
+ import CityPopulationEntityMapper.*
+
+ class CityPopulationEntityMapper extends NCEntityMapper :
+ def map(req: NCRequest, cfg: NCModelConfig, ents:
List[NCEntity]): List[NCEntity] =
+ val cities = ents.filter(_.getId == "city")
+
+ ents.filterNot(_.getId == "city") ++
+ cities ++
+ cities.filter(city => citiesPopulation(city("city:name"))
> 1000000).
+ map(city =>
+ new NCPropertyMapAdapter with NCEntity :
+ override def getTokens: List[NCToken] =
city.getTokens
+ override def getRequestId: String =
req.getRequestId
+ override def getId: String = "big-city"
+ )
+ </pre>
+ <ul>
+ <li>
+ <code>Line 4</code> defines getting cities population data
from some external service.
+ </li>
+ <li>
+ <code>Line 10</code> filters entities by <code>ID</code>.
+ </li>
+ <li>
+ <code>Line 12, 13 and 14</code> define component result
entities set.
+ It contains previously defined <code>city</code> elements, new
elements <code>big-city</code> and
+ another not city elements.
+ </li>
+ </ul>
</section>
<section id="entity-validator">
@@ -167,6 +289,27 @@ id: overview
<p>
You have to implement <a
href="apis/latest/org/apache/nlpcraft/NCEntityValidator.html">NCEntityValidator</a>
trait.
</p>
+ <p>
+ Entities validator is user defined component, where prepared
entities are inspected and exceptions
+ can be thrown from user code to break user input processing.
+ </p>
+
+ <pre class="brush: scala, highlight: [3]">
+ new NCEntityValidator :
+ override def validate(req: NCRequest, cfg: NCModelConfig,
ents: List[NCEntity]): Unit =
+ if ents.exists(_.getId == "restrictedID")
+ then throw new NCException("Sentence cannot be
processed.")
+ </pre>
+
+ <ul>
+ <li>
+ There is anonymous instance of <a
href="apis/latest/org/apache/nlpcraft/NCEntityValidator.html">NCEntityValidator</a>
+ created.
+ </li>
+ <li>
+ <code>Lines 3</code> defines the rule when exception should be
thrown and sentence processing should be stopped.
+ </li>
+ </ul>
</section>
<section id="variant-filter">
@@ -174,6 +317,27 @@ id: overview
<p>
You have to implement <a
href="apis/latest/org/apache/nlpcraft/NCVariantFilter.html">NCVariantFilter</a>
trait.
</p>
+
+ <p>
+ Component which allows filtering detected variants, rejecting
undesirable.
+ </p>
+
+ <pre class="brush: scala, highlight: [3]">
+ new NCVariantFilter :
+ def filter(req: NCRequest, cfg: NCModelConfig, vars:
List[NCVariant]): List[NCVariant] =
+ vars.filter(_.getEntities.exists(_.getId == "requiredID"))
+ </pre>
+
+ <ul>
+ <li>
+ There is anonymous instance of <a
href="apis/latest/org/apache/nlpcraft/NCVariantFilter.html">NCVariantFilter</a>
+ created.
+ </li>
+ <li>
+ <code>Lines 3</code> defines variant's filter,
+ it passed only variants which contain <code>requiredID</code>
elements.
+ </li>
+ </ul>
</section>
</div>
diff --git a/examples/calculator.html b/examples/calculator.html
index 16d7583..081f55c 100644
--- a/examples/calculator.html
+++ b/examples/calculator.html
@@ -252,7 +252,6 @@ fa_icon: fa-cube
<li><a href="#overview">Overview</a></li>
<li><a href="#new_project">New Project</a></li>
<li><a href="#model">Data Model</a></li>
- <li><a href="#code">Model Class</a></li>
<li><a href="#testing">Testing</a></li>
{% include quick-links.html %}
</ul>
diff --git a/installation.html b/installation.html
index 2131048..27d1576 100644
--- a/installation.html
+++ b/installation.html
@@ -22,32 +22,21 @@ id: installation
-->
<div id="installation" class="col-md-8 second-column">
- <section id="tldr">
- <h2 class="section-title">TL;DR; <a href="#"><i class="top-link fas
fa-fw fa-angle-double-up"></i></a></h2>
- <p>
- Watch this quick video (6:17) on how to download, install and
verify your installation:
- </p>
- <div>
- <iframe
- width="514"
- height="289"
-
src="https://www.youtube.com/embed/s4zEVJVhW68?modestbranding=1";
- title="NLPCraft - Download, Install & Verify"
- frameborder="0"
- allow="accelerometer; autoplay; clipboard-write;
encrypted-media; gyroscope; picture-in-picture"
- allowfullscreen>
- </iframe>
- </div>
- </section>
<section id="install">
<h2 class="section-title">Installation <a href="#"><i class="top-link
fas fa-fw fa-angle-double-up"></i></a></h2>
<p>
Apache NLPCraft requires <b>no special installation</b> operations:
</p>
<ul>
- <li>If you <a href="/download.html">downloaded</a> binary ZIP
archive - <a target="asf"
href="https://www.apache.org/dyn/closer.cgi#verify";>verify</a> it and unzip
it</li>
- <li>If you <a href="/download.html">cloned</a> GitHub repository
or downloaded source archive - use <code>mvn clean package -P examples</code>
to build it</li>
- <li>If you <a href="/download.html">used</a> Maven/Gradle/SBT -
you know what you are doing...</li>
+ <li>If you <a href="/download.html">downloaded</a> binary ZIP
archive -
+ <a target="asf"
href="https://www.apache.org/dyn/closer.cgi#verify";>verify</a> it and unzip it.
+ </li>
+ <li>If you <a href="/download.html">cloned</a> GitHub repository
or downloaded source archive -
+ <code>sbt clean compile</code> to build it.
+ </li>
+ <li>Also you can use <code>maven</code> command <code>mvn -B clean
package -P stanford-core,examples</code>
+ and get same result.
+ </li>
</ul>
<p>
You are done 👍
@@ -57,68 +46,55 @@ id: installation
<b>Minimal Java and Scala versions</b>
</p>
<p>
+ Starting with version <b>1.0.0</b> NLPCraft requires JDK 11+
and Scala 3.1.3.<br/>
Starting with version <b>0.9.0</b> NLPCraft requires JDK 11+
and Scala 2.13.<br/>
Starting with version <b>0.7.0</b> NLPCraft requires JDK 11+
and Scala 2.12.<br/>
Prior versions supported JDK 8 and Scala 2.12.
</p>
</div>
</section>
- <section id="clean">
- <h2 class="section-title">Reset Database <a href="#"><i
class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
- <p>
- For releases marked with <i class="fas fa-fw fa-bomb"></i> you
need to reset the existing database. If you are
- using the Apache Ignite storage (default configuration for
NLPCraft) you can accomplish this by
- removing <code>${USER_HOME}/.nlpcraft/ignite</code> folder.
- </p>
- <p>
- If you are using other storage engines (e.g. Postgres or MySQL)
you need to follow their specific
- procedures to recreate the database using SQL scripts from
<code>sql</code> sub-folder in NLPCraft
- installation directory.
- </p>
- </section>
- <section id="config">
- <h2 class="section-title">External Configuration <a href="#"><i
class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
- <p>
- Due to <a target="asf"
href="https://apache.org/legal/release-policy.html";>Apache release policy</a>
the official Apache release cannot include content that is based on
- <a target=_
href="https://creativecommons.org/licenses/by/4.0/";>Commons Attribution 4.0
International</a> (CC BY 4.0) license
- or any binary artifacts regardless of their respective licences.
Due to these restrictions the official Apache NLPCraft release
- does not include certain configuration files that are necessary
for its operation. These files will be auto-downloaded upon first star of
NLPCraft (hence the internet
- connection is required for the first start).
- On subsequent starts the NLPCraft will check for the updates and
will re-download these files only if they were changed.
- </p>
+ <section id="new_project">
+ <h2 class="section-title">Create New Project <a href="#"><i
class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
<p>
- By default, this external configuration is hosted on GitHub (in
<code>external_config</code> branch of NLPCraft project) from where it will
- be automatically downloaded on the first start: <a target="github"
href="https://github.com/apache/incubator-nlpcraft/raw/external_config/external";>https://github.com/apache/incubator-nlpcraft/raw/external_config/external</a>.
- You can change this location, e.g. move the external configuration
behind your firewall or onto your local file system,
- as well as change other parameters related to auto-download of the
external <a href="/server-and-probe.html#config">configuration</a>
- in <code>*.conf</code> file (see <code>nlpcraft.extConfig</code>
property).
+ You can create new Scala projects in many ways - we'll use SBT
+ to accomplish this task. Make sure that <code>build.sbt</code>
file has the following content:
</p>
- </section>
- <section id="ignite">
- <h2 class="section-title">Apache Ignite 2.x <span
class="amp">&</span> Java 11+ <a href="#"><i class="top-link fas fa-fw
fa-angle-double-up"></i></a></h2>
+ <pre class="brush: js, highlight: []">
+ ThisBuild / version := "0.1.0-SNAPSHOT"
+ ThisBuild / scalaVersion := "3.1.3"
+ lazy val root = (project in file("."))
+ .settings(
+ name := "Your project",
+ version := "{{site.latest_version}}",
+ libraryDependencies += "org.apache.nlpcraft" % "nlpcraft" %
"{{site.latest_version}}",
+ libraryDependencies += "org.scalatest" %% "scalatest" %
"3.2.14" % "test"
+ )
+ </pre>
+
+ <ul>
+ <li>
+ Now you can use <code>NlpCraft</code> library in <code>Your
project</code>.
+ </li>
+ <li>
+ You can use <code>Sbt</code> or <code>Maven</code> systems
depending on your requirements via links
+ to <a href="https://search.maven.org/";>Maven Central</a>.
+ </li>
+ <li>
+ You also can build <code>NlpCraft</code> library soource code
and use references on jar files via <code>Ant</code>
+ or any another way.
+ </li>
+ </ul>
+
<p>
- Starting with NLPCraft 0.7.0, NLPCraft requires JDK 11 as its
minimum JDK version.
- NLPCraft uses <a target=_ href="https://ignite.apache.org/";>Apache
Ignite 2.x</a> project as its
- underlying in-memory distributed storage and cluster computing
framework. When running Apache Ignite 2.x
- on Java 11 you need to follow these
- <a target=_
href="https://apacheignite.readme.io/docs/getting-started#section-running-ignite-with-java-9-10-11";>instructions</a>
which
- are automatically followed when you use <a
href="/tools/script.html"><code>nlpcraft.{sh|cmd}</code></a> script.
+ Go to Examples section to start work with concrete example.
</p>
- <div class="bq info">
- <p>
- <b>NOTE:</b> It is NOT recommended to change the version of
Apache Ignite inside NLPCraft project.
- </p>
- </div>
- <br/>
</section>
-</div>
+ </div>
<div class="col-md-2 third-column">
<ul class="side-nav">
<li class="side-nav-title">On This Page</li>
<li><a href="#install">Installation</a></li>
- <li><a href="#clean">Reset Database</a></li>
- <li><a href="#config">External Config</a></li>
- <li><a href="#ignite">Apache Ignite</a></li>
+ <li><a href="#new_project">New Project</a></li>
{% include quick-links.html %}
</ul>
</div>
diff --git a/semantic.html b/semantic.html
new file mode 100644
index 0000000..bfde0c5
--- /dev/null
+++ b/semantic.html
@@ -0,0 +1,627 @@
+---
+active_crumb: Docs
+layout: documentation
+id: semantic
+---
+
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<div class="col-md-8 second-column">
+ <section id="overview">
+ <h2 class="section-title">Semantic model<a href="#"><i class="top-link
fas fa-fw fa-angle-double-up"></i></a></h2>
+
+ <p>
+ Semantic entity parser <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">NCSemanticEntityParser</a>
+ is the implementation of <a
href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
+ which deserves a special mention.
+ It defines list of <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
+ which are represent <a
href="https://en.wikipedia.org/wiki/Named-entity_recognition";>Named
entities</a>.
+ We will name this list as <code>Semantic Model</code>.
+
+ Let's talk a little bit more about <a
href="https://en.wikipedia.org/wiki/Named-entity_recognition";>Named
entities</a>.
+ </p>
+ </section>
+
+ <section id="ne">
+ <h2 class="section-title">Named Entities<a href="#"><i class="top-link
fas fa-fw fa-angle-double-up"></i></a></h2>
+ <p>
+ Named entity, also known as a semantic element or a token, is one
of the main a components defined by the NLPCraft data model.
+ A named entity is one or more individual words that have a
consistent semantic meaning and typically denote a
+ real-world object, such as persons, locations, number, date and
time, organizations, products, etc. Such
+ object can be abstract or have a physical existence.
+ </p>
+ <p>
+ For example, in the following sentence: TODO: PIC
+ </p>
+ <figure>
+ <img alt="named entities" class="img-fluid"
src="/images/named-entities.png">
+ <figcaption><b>Fig 2.</b> Named Entities</figcaption>
+ </figure>
+ <p>
+ the following named entities can be detected:
+ </p>
+ <table class="gradient-table">
+ <thead>
+ <tr>
+ <th>Words</th>
+ <th>Type</th>
+ <th>Normalized Value</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><b>Top 20</b></td>
+ <td><code>user:element:1</code></td>
+ <td>top 20</td>
+ </tr>
+ <tr>
+ <td><b>best pages</b></td>
+ <td><code>user:element:2</code></td>
+ <td>best pages</td>
+ </tr>
+ <tr>
+ <td><b>California USA</b></td>
+ <td><code>stanford:city</code></td>
+ <td>USA, California</td>
+ </tr>
+ <tr>
+ <td><b>last 3 months</b></td>
+ <td><code>stanford:date</code></td>
+ <td>1/1/2021 - 4/1/2021</td>
+ </tr>
+ </tbody>
+ </table>
+ <p>
+ In most cases named entities will have associated <em>normalized
value</em>. It is especially important for named entities that have many
+ notational forms such as time and date, currency, geographical
locations, etc. For example, <code>New York</code>,
+ <code>New York City</code> and <code>NYC</code> all refer to the
same "New York City, NY USA" location which is a standard normalized form.
+ </p>
+ <p>
+ The process of detecting named entities is called Named Entity
Recognition (NER). There are many ways of how a certain named entity can be
detected: through list of synonyms, by name, rule-based or by using
+ statistical techniques like neural networks with large corpus of
predefined data. NLPCraft natively supports synonym-based
+ named entities definition as well as the ability to compose new
named entities through powerful <a href="/intent-matching.html">Intent
Definition Language</a> (IDL)
+ combining other named entities including named entities from
+ such OpenNLP, or Stanford CoreNLP, look at the <a
href="built-components.html">Built component</a> chapter.
+ </p>
+ <p>
+ Named entities allow you to abstract from basic linguistic forms
like nouns and verbs to deal with the higher level semantic
+ abstractions like geographical location or time when you are
trying to understand the meaning of the sentence.
+ One of the main goals of named entities is to act as an input
ingredients for <a href="/intent-matching.html">intent matching</a>.
+ </p>
+ <div class="bq info">
+ <p>
+ <b>😀 User Input → Named Entities → Parsing Variants → Intent
Matcher → Winning Intent 🚀</b>
+ </p>
+ <p>
+ User input is parsed into the list of named entities. That
list is then further transformed into one or more
+ parsing variants where each variant represents a particular
order and combination of detected named entities.
+ Finally, the list of variants act as an input to intent
matching where each variant is matched against every intent
+ in the process of detecting the best matching intent for the
original user input.
+ </p>
+ </div>
+ </section>
+
+ <section id="elements">
+ <h2 class="section-title">Semantic elements<a href="#"><i
class="top-link fas fa-fw fa-angle-double-up"></i></a></h2>
+ <p>
+ <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
represents
+ NER element for its detection un the user input.
+ <p>
+
+ <div class="bq info">
+ <p>
+ <b>Semantic Element <span class="amp">&</span> Named Entity
<span class="amp">&</span> Token</b>
+ </p>
+ <p>
+ Terms 'semantic element', 'named entity' and 'token' are used
throughout this documentation relatively interchangeably:
+ </p>
+ <dl>
+ <dt>Semantic Element</dt>
+ <dd>
+ Denotes a named entity <em>declared</em> in NLPCraft model.
+ </dd>
+ <dt>Token</dt>
+ <dd>
+ Denotes a semantic element that was <em>detected</em> by
NLPCraft in the user input.
+ </dd>
+ <dt>Named Entity</dt>
+ <dd>
+ Denotes a classic term, i.e. one or more individual words
that have a
+ consistent semantic meaning and typically define a
real-world object.
+ </dd>
+ </dl>
+ </div>
+
+ <p>
+ Each <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
+ is presented by <code>id</code>, <code>groups</code>,
<code>synonyms</code>, <code>values</code> and <code>properties</code>.
+ <p>
+ <span id="synonyms" class="section-sub-title">Synonyms <a href="#"><i
class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ NLPCraft uses fully deterministic named entity recognition and is
not based on statistical approaches that
+ would require pre-existing marked up data sets and extensive
training. For each semantic element you can either provide a
+ set of synonyms to match on or specify a piece of code that would
be responsible for detecting that named
+ entity (discussed below). A synonym can have one or more
individual words. Note that element's ID is its
+ implicit synonym so that even if no additional synonyms are
defined at least one synonym always exists. Note
+ also that synonym matching is performed on <em>normalized</em> and
<em>stemmatized</em> forms of both
+ a synonym and user input.
+ </p>
+
+ <p>
+ Here's an example of a simple semantic element definition in JSON:
+ </p>
+ <pre class="brush: js, highlight: [6,7,8,9,10,11,12]">
+ ...
+ "elements": [
+ {
+ "id": "transport.vehicle",
+ "description": "Transportation vehicle",
+ "synonyms": [
+ "car",
+ "truck",
+ "light duty truck"
+ "heavy duty truck"
+ "sedan",
+ "coupe"
+ ]
+ }
+ ]
+ ...
+ </pre>
+ <p>
+ While adding multi-word synonyms looks somewhat
+ trivial - in real models, the naive approach can lead to thousands
and even tens of thousands of
+ possible synonyms due to words, grammar, and linguistic
permutations - which quickly becomes untenable if
+ performed manually.
+ </p>
+ <p>
+ NLPCraft provides an effective tool for a compact synonyms
representation. Instead of listing all possible
+ multi-word synonyms one by one you can use combination of
following techniques:
+ </p>
+ <ul>
+ <li><a href="#macros">Macros</a></li>
+ <li><a href="#regex">Regular expressions</a></li>
+ <li><a href="#option-groups">Option Groups</a></li>
+ </ul>
+ <p>
+ Each whitespace separated string in the synonym can be either a
regular word (like in the above transportation example
+ where it will be matched on using its normalized and stemmatized
form) or one of the above expression.
+ </p>
+ <p>
+ Note that this synonyms definition is also used in the following
+ <a
href="/apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
methods:
+ </p>
+ <ul>
+ <li><code>getSynonyms()</code> - gets synonyms to match on.</li>
+ <li><code>getValues()</code> - get values to match on (see <a
href="#values">below</a>).</li>
+ </ul>
+ <span id="values" class="section-sub-title">Element Values <a
href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Semantic element can have an optional set of special synonyms
called <em>values</em> or "proper nouns" for this element.
+ Unlike basic synonyms, each value is a pair of a name and a set of
standard synonyms by which that value,
+ and ultimately its element, can be recognized in the user input.
Note that the value name itself acts as an
+ implicit synonym even when no additional synonyms added for that
value.
+ </p>
+ <p>
+ When a semantic element is recognized it is made available to the
model's matching logic as an instance of
+ the <a
href="/apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a> interface.
+ This interface has a method
+ <a
href="/apis/latest/org/apache/nlpcraft/NCToken.html">getValue()</a> which
+ returns the name of the value, if any, by which
+ that semantic element was recognized. That value name can be
further used in intent matching.
+ </p>
+ <p>
+ To understand the importance of the values consider the following
changes to our transportation
+ example model:
+ </p>
+ <pre class="brush: js, highlight:
[19,20,21,22,23,24,25,26,27,28,29,30]">
+ ...
+ "macros": [
+ {
+ "name": "<TRUCK_TYPE>",
+ "macro": "{light duty|heavy duty|half ton|1/2 ton|3/4
ton|one ton|super duty}"
+ }
+ ]
+ "elements": [
+ {
+ "id": "transport.vehicle",
+ "description": "Transportation vehicle",
+ "synonyms": [
+ "car",
+ "{<TRUCK_TYPE>|_} {pickup|_} truck"
+ "sedan",
+ "coupe"
+ ],
+ "values": [
+ {
+ "value": "mercedes",
+ "synonyms": ["mercedes-ben{z|s}", "mb", "ben{z|s}"]
+ },
+ {
+ "value": "bmw",
+ "synonyms": ["{bimmer|bimer|beemer}", "bayerische
motoren werke"]
+ }
+ {
+ "value": "chevrolet",
+ "synonyms": ["chevy"]
+ }
+ ]
+ }
+ ]
+ ...
+ </pre>
+ <p>
+ With that setup <code>transport.vehicle</code> element will be
recognized by any of the following input string:
+ </p>
+ <ul>
+ <li><code>car</code></li>
+ <li><code>benz</code> (with value <code>mercedes</code>)</li>
+ <li><code>3/4 ton pickup truck</code></li>
+ <li><code>light duty truck</code></li>
+ <li><code>chevy</code> (with value <code>chevrolet</code>)</li>
+ <li><code>bimmer</code> (with value <code>bmw</code>)</li>
+ <li><code>transport.vehicle</code></li>
+ </ul>
+ <span id="groups" class="section-sub-title">Element Groups <a
href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Each semantic element always belongs to one or more groups.
Semantic element provides its groups via
+ <a
href="/apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">getGroups()</a>
method.
+ By default, if element group is not specified, the element ID will
act as its default group ID.
+ Group membership is a quick and easy way to organise similar
semantic elements together and use this
+ categorization in <a href="/intent-matching.html">IDL</a> intents.
+ </p>
+ <p>
+ Note that the proper grouping of the elements is also necessary
for the correct operation of
+ Short-Term-Memory (STM) in the conversational context. Consider a
+ <a
href="/apis/latest/org/apache/nlpcraft/NCToken.html">NCToken</a> that
+ represents a previously found semantic element that is stored in
the conversation. Such token
+ will be overridden in the conversation by the more <b>recent
token</b>
+ from the <b>same group</b> - a critical rule of maintaining the
proper conversational context.
+ See
+ <a
href="/apis/latest/org/apache/nlpcraft/NCConversation.html">NCConversation</a>
+ for mode details.
+ </p>
+ </section>
+
+ <section id="macros">
+ <h2 class="section-title">Macros <a href="#"><i class="top-link fas
fa-fw fa-angle-double-up"></i></a></h2>
+ <p>
+ Listing all possible multi-word synonyms for a given element can
be a time-consuming task. Macros
+ together with option groups allow for significant simplification
of this task.
+ Macros allow you to give a name to an often used set of words or
option groups and reuse it without
+ repeating those words or option groups again and again. A model
provides a list of macros via
+ <a
href="/apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticEntityParser.html">macros</a>
method.
+ Each macro has a name in a form of <code><X></code> where
<code>X</code>
+ is any string, and a string value. Note that macros can be nested
(but not recursive), i.e. macro value can include
+ references to other macros. When macro name <code>X</code> is
encountered in the synonym it gets recursively
+ replaced with its value.
+ </p>
+ <p>
+ Here's a code snippet of macro definitions using JSON definition:
+ </p>
+ <pre class="brush: js">
+ "macros": [
+ {
+ "name": "<A>",
+ "macro": "aaa"
+ },
+ {
+ "name": "<B>",
+ "macro": "<A> bbb"
+ },
+ {
+ "name": "<C>",
+ "macro": "<A> bbb {z|w}"
+ }
+ ]
+ </pre>
+ <span id="option-groups" class="section-sub-title">Option Groups <a
href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Option groups are similar to wildcard patterns that operates on a
single word base. One line of
+ option group expands into one or more individual synonyms. Option
groups is the key mechanism for shortened
+ synonyms notation. The following examples demonstrate how to use
option groups.
+ </p>
+ <p>
+ Consider the following macros defined below (note that macros
<code><B></code> and <code><C></code>
+ are nested):
+ </p>
+ <table class="gradient-table">
+ <thead>
+ <tr>
+ <th>Name</th>
+ <th>Value</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code><A></code></td>
+ <td><code>aaa</code></td>
+ </tr>
+ <tr>
+ <td><code><B></code></td>
+ <td><code><A> bbb</code></td>
+ </tr>
+ <tr>
+ <td><code><C></code></td>
+ <td><code><A> bbb {z|w}</code></td>
+ </tr>
+ </tbody>
+ </table>
+ <p>
+ Then the following option group expansions will occur in these
examples:
+ </p>
+ <table class="gradient-table">
+ <thead>
+ <tr>
+ <th>Synonym</th>
+ <th>Synonym Expansions</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><code><A> {b|_} c</code></td>
+ <td>
+ <code>"aaa b c"</code><br>
+ <code>"aaa c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code><A> {b|a}[1,2] c</code></td>
+ <td>
+ <code>"aaa b c"</code><br>
+ <code>"aaa b b c"</code><br>
+ <code>"aaa a c"</code><br>
+ <code>"aaa a a c"</code><br>
+ <code>"aaa c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td>
+ <code><B> {b|_} c</code><br>
+ or<br>
+ <code><B> {b}[0,1] c</code>
+ </td>
+ <td>
+ <code>"aaa bbb b c"</code><br>
+ <code>"aaa bbb c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>{b|\{\_\}}</code></td>
+ <td>
+ <code>"b"</code><br>
+ <code>"b {_}"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>a {b|_}. c</code></td>
+ <td>
+ <code>"a b. c"</code><br>
+ <code>"a . c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>a .{b, |_}. c</code></td>
+ <td>
+ <code>"a .b, . c"</code><br>
+ <code>"a .. c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>
+ {% raw %}a {{b|c}|_}.{% endraw %}</code></td>
+ <td>
+ <code>"a ."</code><br>
+ <code>"a b."</code><br>
+ <code>"a c."</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>a {% raw %}{{{<C>}}|{_}}{% endraw %}
c</code></td>
+ <td>
+ <code>"a aaa bbb z c"</code><br>
+ <code>"a aaa bbb w c"</code><br>
+ <code>"a c"</code>
+ </td>
+ </tr>
+ <tr>
+ <td><code>{% raw %}{{{a}}} {b||_|{{_}}||_}{% endraw
%}</code></td>
+ <td>
+ <code>"a b"</code><br>
+ <code>"a"</code>
+ </td>
+ </tr>
+ </tbody>
+ </table>
+ <p>
+ Specifically:
+ </p>
+ <ul>
+ <li><code>{A|B}</code> denotes either <code>A</code> or
<code>B</code>.</li>
+ <li>
+ <code>{A|B|_}</code> denotes either <code>A</code> or
<code>B</code> or nothing.
+ <ul>
+ <li>Symbol <code>_</code> cam appear anywhere in the list
of options, i.e. <code>{A|B|_}</code> is equal to <code>{A|_|B}</code>.</li>
+ </ul>
+ </li>
+ <li>
+ <code>{C}[x,y]</code> denotes an option group with quantifier,
i.e. group <code>C</code> appearing from <code>x</code> to <code>y</code> times
inclusive.
+ <ul>
+ <li>For example, <code>{C}[1,3]</code> is the same as
<code>{C|C C|C C C}</code> notation.</li>
+ <li>Note that <code>{C|_}</code> is equal to
<code>{C}[0,1]</code></li>
+ </ul>
+ </li>
+ <li>Excessive curly brackets are ignored, when safe to do so.</li>
+ <li>Macros cannot be recursive but can be nested.</li>
+ <li>Option groups can be nested.</li>
+ <li>
+ <code>'\'</code> (backslash) can be used to escape
<code>'{'</code>, <code>'}'</code>, <code>'|'</code> and
+ <code>'_'</code> special symbols used by the option groups.
+ </li>
+ <li>Excessive whitespaces are trimmed when expanding option
groups.</li>
+ </ul>
+ <p>
+ We can rewrite our transportation semantic element in a more
efficient way using macros and option groups.
+ Even though the actual length of definition hasn't changed much it
now auto-generates many dozens of synonyms
+ we would have to write out manually otherwise:
+ </p>
+ <pre class="brush: js, highlight: [4,5,14]">
+ ...
+ "macros": [
+ {
+ "name": "<TRUCK_TYPE>",
+ "macro": "{ {light|super|heavy|medium} duty|half ton|1/2
ton|3/4 ton|one ton}"
+ }
+ ]
+ "elements": [
+ {
+ "id": "transport.vehicle",
+ "description": "Transportation vehicle",
+ "synonyms": [
+ "car",
+ "{<TRUCK_TYPE>|_} {pickup|_} truck"
+ "sedan",
+ "coupe"
+ ]
+ }
+ ]
+ ...
+ </pre>
+ <span id="regex" class="section-sub-title">Regular Expressions <a
href="#"><i class="top-link fas fa-fw fa-angle-double-up"></i></a></span>
+ <p>
+ Any individual synonym word that starts and ends with
<code>//</code> (two forward slashes) is
+ considered to be Java regular expression as defined in
<code>java.util.regex.Pattern</code>. Note that
+ regular expression can only span a single word, i.e. only
individual words from the user input will be
+ matched against given regular expression and no whitespaces are
allowed within regular expression. Note
+ also that option group special symbols <code>{</code>,
<code>}</code>,
+ <code>|</code> and <code>_</code> have to be escaped in the
regular expression using <code>\</code>
+ (backslash).
+ </p>
+ <p>
+ For example, the following synonym:
+ </p>
+ <pre class="brush: js">
+ "synonyms": [
+ "{foo|//[bar].+//}}"
+ ]
+ </pre>
+ <p>
+ will match word <code>foo</code> or any other strings that start
with <code>bar</code> as long as
+ this string doesn't contain whitespaces.
+ </p>
+ <div class="bq info">
+ <b>Regular Expressions Performance</b>
+ <p>
+ It's important to note that regular expressions can
significantly affect the performance of the
+ NLPCraft processing if used uncontrolled. Use it with caution
and test the performance
+ of your model to ensure it meets your requirements.
+ </p>
+ </div>
+ </section>
+
+ <section id="usage">
+ <h2 class="section-title">Usage examples<a href="#"><i class="top-link
fas fa-fw fa-angle-double-up"></i></a></h2>
+
+ <p>
+ The following examples shows how to build model programmatically.
+ </p>
+
+ <pre class="brush: scala, highlight: [3, 5, 10]">
+ val mdl = new NCModelAdapter(
+ NCModelConfig("test.id", "Test Model", "1.0"),
+ new NCPipelineBuilder().withSemantic(
+ "en",
+ Map(
+ "<OF>" -> "{of|for|per}",
+ "<CUR>" -> "current|present|now|local}",
+ "<TIME>" -> "{time <OF> day|day
time|date|time|moment|datetime|hour|o'clock|clock|date time|date and time|time
and date}",
+ )
+ List(
+ new NCSemanticElement():
+ override def getId: String = "time"
+ override def getSynonyms: Set[String] =
Set("{<CUR>|_} <TIME>", "what <TIME> {is it now|now|is it|_}"
)
+ )
+ ).build
+ )
+ </pre>
+ <ul>
+ <li>
+ <code>Line 5</code> shows <code>macro</code> parameter usage.
+ </li>
+ <li>
+ <code>Line 10</code> shows <code>macro</code> list of <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCSemanticElement.html">NCSemanticElement</a>
parameter usage.
+ </li>
+ <li>
+ Note that usage <code>withSemantic()</code> method which
represented on <code>line 3</code> i optional.
+ You can add <a
href="apis/latest/org/apache/nlpcraft/nlp/parsers/NCNLPEntityParser.html">NCNLPEntityParser</a>
+ as usual <a
href="apis/latest/org/apache/nlpcraft/NCEntityParser.html">NCEntityParser</a>
+ when you define your <a
href="apis/latest/org/apache/nlpcraft/NCPipeline.html">NCPipeline</a>.
+ </li>
+ </ul>
+
+ <p>
+ The following examples is based on YAML semantic elements
representation.
+ </p>
+
+ <pre class="brush: js, highlight: []">
+ macros:
+ "<OF>": "{of|for|per}"
+ "<CUR>": "{current|present|now|local}"
+ "<TIME>": "{time <OF> day|day
time|date|time|moment|datetime|hour|o'clock|clock|date time|date and time|time
and date}"
+ elements:
+ - id: "x:time"
+ description: "Date and/or time token indicator."
+ synonyms:
+ - "{<CUR>|_} <TIME>"
+ - "what <TIME> {is it now|now|is it|_}"
+ </pre>
+
+ <ul>
+ <li>
+ Same macros and the same element as in previous example are
defined here in
+ <code>time_model.yaml</code> YAML file.
+ </li>
+ </ul>
+ <pre class="brush: scala, highlight: [3]">
+ val mdl = new NCModelAdapter(
+ NCModelConfig("test.id", "Test Model", "1.0"),
+ new NCPipelineBuilder().withSemantic("en",
"time_model.yaml").build
+ )
+ </pre>
+ <ul>
+ <li>
+ <code>Line 3</code> makes semantic model which defined in
<code>time_model.yaml</code> YAML file.
+ </li>
+ </ul>
+ </section>
+</div>
+<div class="col-md-2 third-column">
+ <ul class="side-nav">
+ <li class="side-nav-title">On This Page</li>
+ <li><a href="#overview">Overview</a></li>
+ <li><a href="#ne">Named Entities</a></li>
+ <li><a href="#elements">Semantic elements</a></li>
+ <li><a href="#macros">Macros</a></li>
+ <li><a href="#usage">Usage examples</a></li>
+ {% include quick-links.html %}
+ </ul>
+</div>
+
+
+
+
