This is an automated email from the ASF dual-hosted git repository.
riemer pushed a commit to branch dev
in repository
https://gitbox.apache.org/repos/asf/incubator-streampipes-website.git
The following commit(s) were added to refs/heads/dev by this push:
new b1f02ac [STREAMPIPES-197] Add concept overview page
b1f02ac is described below
commit b1f02acac19c1f2ffec9c59a7abbf2fde4de1b34
Author: Dominik Riemer <[email protected]>
AuthorDate: Tue Apr 27 11:30:42 2021 +0200
[STREAMPIPES-197] Add concept overview page
---
documentation/docs/02_concepts-overview.md | 35 +++++++++++++++++++++
documentation/docs/02_concepts_overview.md | 7 -----
documentation/website/i18n/en.json | 8 ++---
documentation/website/sidebars.json | 6 +---
documentation/website/static/css/custom.css | 3 +-
.../img/02_concepts-overview/01_overview.png | Bin 0 -> 28474 bytes
6 files changed, 42 insertions(+), 17 deletions(-)
diff --git a/documentation/docs/02_concepts-overview.md
b/documentation/docs/02_concepts-overview.md
new file mode 100644
index 0000000..f7e9f8f
--- /dev/null
+++ b/documentation/docs/02_concepts-overview.md
@@ -0,0 +1,35 @@
+---
+id: concepts-overview
+title: StreamPipes Concepts
+sidebar_label: Overview
+---
+
+To understand how StreamPipes works, the knowledge of a few core concepts,
illustrated below, will be helpful.
+
+<img class="docs-image" src="/docs/img/02_concepts-overview/01_overview.png"
alt="Overview of concepts">
+
+## Adapter
+An adapter connects to any external data source and forwards received events
to the internal StreamPipes system. Within StreamPipes, the output of adapters
are available in form of the two primary building blocks **Data Set** and
**Data Stream**.
+Adapters can be either created by using StreamPipes Connect, a module to
easily connect to new data sources directly from the user interface, or by
defining an adapter using the provided Software Development Kit (SDK).
+
+## Data Set / Data Stream
+**Data Streams** and **Data Sets** represent the primary source for working
with events in StreamPipes.
+A stream is an ordered sequence of events, where an event typically consists
of one or more observation values and additional metadata. The "structure" (or
schema) of an event provided by a data stream or set is stored in the internal
semantic schema registry of StreamPipes.
+While data streams are typically unbounded, data sets have a fixed end and are
internally "replayed" by the system from beginning to end once they are used as
part of a pipeline.
+As follows, although when referring to data streams, most concepts also apply
for data sets.
+
+## Data Processor
+**Data Processors** in StreamPipes transform one or more input data streams
into an output data stream.
+Such transformations can be rather simple, e.g. filtering based on a
predefined rule or more complex, e.g. applying rule-based or learning-based
algorithms on the data.
+Data Processors can be applied on any data stream that matches the input
requirements of a processor. In addition, most processors can be configured by
providing user-defined parameters directly in the user interface.
+Processing elements define stream requirements that are a set of minimum
properties an incoming event stream must provide. Data processors can keep
state or perform stateless operations.
+At runtime, data streams are processed by using one of the underlying runtime
wrappers (see the developer guide for more details).
+
+## Data Sink
+**Data Sinks** consume event streams similar to Data Processors, but do not
provide an output data stream. As such, data sinks typically perform some
action or trigger a visualization as a result of a stream transformation.
+Similar to data processors, sinks also require for the presence of specific
input requirements of any bound data stream and can be customized.
+StreamPipes provides several internal data sinks, e.g., to create
notifications, visualize live data or persist historical data of incoming
streams. In addition, various data sinks are provided to forward data streams
to external systems such as databases.
+
+## Pipeline
+A pipeline in Apache StreamPipes describes the transformation process from a
data stream to a data sink. Typically, a pipeline consists of at least one data
stream (or data set), zero or more data processors and at least one data sink.
+Pipelines are built by users in a graphical way using the **Pipeline Editor**
and can be started and stopped at any time.
diff --git a/documentation/docs/02_concepts_overview.md
b/documentation/docs/02_concepts_overview.md
deleted file mode 100644
index db9f241..0000000
--- a/documentation/docs/02_concepts_overview.md
+++ /dev/null
@@ -1,7 +0,0 @@
----
-id: concepts-overview
-title: StreamPipes Concepts
-sidebar_label: Overview
----
-
-tbd
diff --git a/documentation/website/i18n/en.json
b/documentation/website/i18n/en.json
index a836b8d..37c5eb0 100644
--- a/documentation/website/i18n/en.json
+++ b/documentation/website/i18n/en.json
@@ -13,10 +13,6 @@
"title": "Interactive Tutorial",
"sidebar_label": "Interactive Tutorial"
},
- "concepts-overview": {
- "title": "StreamPipes Concepts",
- "sidebar_label": "Overview"
- },
"concepts-adapter": {
"title": "Data Adapters",
"sidebar_label": "Data Adapters"
@@ -29,6 +25,10 @@
"title": "Glossary",
"sidebar_label": "Glossary"
},
+ "concepts-overview": {
+ "title": "StreamPipes Concepts",
+ "sidebar_label": "Overview"
+ },
"concepts-pipelines": {
"title": "Pipelines",
"sidebar_label": "Pipelines"
diff --git a/documentation/website/sidebars.json
b/documentation/website/sidebars.json
index 40073e3..625a0f6 100644
--- a/documentation/website/sidebars.json
+++ b/documentation/website/sidebars.json
@@ -5,11 +5,7 @@
"try-tutorial"
],
"\uD83D\uDCA1 Concepts": [
- "concepts-overview",
- "concepts-adapter",
- "concepts-data-streams",
- "concepts-pipeline",
- "concepts-glossary"
+ "concepts-overview"
],
"\uD83C\uDF93 Use StreamPipes": [
"use-connect",
diff --git a/documentation/website/static/css/custom.css
b/documentation/website/static/css/custom.css
index 5acbdcd..0747b2f 100644
--- a/documentation/website/static/css/custom.css
+++ b/documentation/website/static/css/custom.css
@@ -23,7 +23,8 @@
@media only screen and (min-width: 1024px) {
.separateOnPageNav .wrapper, .separateOnPageNav .headerWrapper.wrapper {
- max-width: 1600px;
+ max-width: 1400px;
+ }
}
@media only screen and (max-width: 1023px) {
diff --git
a/documentation/website/static/img/02_concepts-overview/01_overview.png
b/documentation/website/static/img/02_concepts-overview/01_overview.png
new file mode 100644
index 0000000..77fcad6
Binary files /dev/null and
b/documentation/website/static/img/02_concepts-overview/01_overview.png differ
