This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-hop.git
The following commit(s) were added to refs/heads/master by this push:
new 0012816 HOP-3267 descriptions for remaining user manual pages, tech
manual
new 2bb7517 Merge pull request #1064 from bamaer/HOP-3267
0012816 is described below
commit 0012816354b985c38552460dd0b2981c5c21b000
Author: Bart Maertens <[email protected]>
AuthorDate: Wed Sep 15 20:50:21 2021 +0200
HOP-3267 descriptions for remaining user manual pages, tech manual
---
.../modules/ROOT/pages/docker-container.adoc | 2 ++
.../ROOT/pages/hop-vs-kettle/hop-vs-kettle.adoc | 2 ++
.../ROOT/pages/hop-vs-kettle/if-you-know-kettle.adoc | 7 ++++---
.../pages/hop-vs-kettle/import-kettle-projects.adoc | 3 ++-
.../modules/ROOT/pages/hop-vs-kettle/index.adoc | 2 ++
.../images/hop-gui/pipeline/failed-pipeline.png | Bin 6513 -> 48382 bytes
.../hop-user-manual/modules/ROOT/pages/concepts.adoc | 1 +
.../hop-user-manual/modules/ROOT/pages/hop-usps.adoc | 2 ++
docs/hop-user-manual/modules/ROOT/pages/index.adoc | 1 +
.../modules/ROOT/pages/pipeline/create-pipeline.adoc | 18 +++++++++++-------
.../modules/ROOT/pages/pipeline/errorhandling.adoc | 1 +
.../ROOT/pages/pipeline/hop-pipeline-editor.adoc | 5 +++++
.../ROOT/pages/pipeline/metadata-injection.adoc | 3 ++-
.../modules/ROOT/pages/pipeline/partitioning.adoc | 1 +
.../ROOT/pages/pipeline/pipeline-unit-testing.adoc | 1 +
.../modules/ROOT/pages/pipeline/pipelines.adoc | 5 +++++
.../pages/pipeline/run-preview-debug-pipeline.adoc | 4 +++-
.../ROOT/pages/plugins/import/kettle-import.adoc | 3 ++-
.../modules/ROOT/pages/protips/index.adoc | 1 +
.../modules/ROOT/pages/samples/samples.adoc | 1 +
.../modules/ROOT/pages/technology/azure/index.adoc | 3 ++-
.../ROOT/pages/technology/cassandra/index.adoc | 2 ++
.../modules/ROOT/pages/technology/google/index.adoc | 10 ++++++++++
.../modules/ROOT/pages/technology/neo4j/index.adoc | 1 +
.../technology/neo4j/working-with-neo4j-data.adoc | 2 ++
.../modules/ROOT/pages/technology/technology.adoc | 6 ++++--
.../modules/ROOT/pages/variables.adoc | 1 +
docs/hop-user-manual/modules/ROOT/pages/vfs.adoc | 1 +
.../modules/ROOT/pages/vfs/aws-s3-vfs.adoc | 1 +
.../ROOT/pages/vfs/azure-blob-storage-vfs.adoc | 1 +
.../modules/ROOT/pages/vfs/dropbox-vfs.adoc | 1 +
.../ROOT/pages/vfs/google-cloud-storage-vfs.adoc | 1 +
.../modules/ROOT/pages/vfs/google-drive-vfs.adoc | 1 +
.../modules/ROOT/pages/workflow/actions.adoc | 6 +++++-
.../modules/ROOT/pages/workflow/create-workflow.adoc | 5 +++--
.../ROOT/pages/workflow/run-debug-workflow.adoc | 1 +
.../native-local-workflow-engine.adoc | 4 +++-
.../native-remote-workflow-engine.adoc | 2 ++
.../workflow-run-configurations.adoc | 2 ++
.../modules/ROOT/pages/workflow/workflows.adoc | 3 +++
40 files changed, 96 insertions(+), 21 deletions(-)
diff --git a/docs/hop-tech-manual/modules/ROOT/pages/docker-container.adoc
b/docs/hop-tech-manual/modules/ROOT/pages/docker-container.adoc
index ad46fa2..7e169a8 100644
--- a/docs/hop-tech-manual/modules/ROOT/pages/docker-container.adoc
+++ b/docs/hop-tech-manual/modules/ROOT/pages/docker-container.adoc
@@ -14,6 +14,8 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Apache Hop provides a Docker image for long (Hop Server) and
short-lived (hop-run) containers. An additional image is available for Hop Web.
Both images are available on Docker Hub.
+
[[DockerContainer-DockerContainer]]
= Docker container
diff --git
a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/hop-vs-kettle.adoc
b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/hop-vs-kettle.adoc
index 7d91261..486a554 100644
--- a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/hop-vs-kettle.adoc
+++ b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/hop-vs-kettle.adoc
@@ -14,6 +14,7 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Apache Hop is an independent platform that originated from the
same code base as Kettle (Pentaho Data Integration). This page provides an
overview of the differences in concepts, configuration, engines and features
between Hop and Kettle/PDI
[[HopvsKettle]]
:imagesdir: ../assets/images
:openvar: ${
@@ -22,6 +23,7 @@ under the License.
= Hop vs Kettle - A Quick Comparison
+
== Concepts
[width=90%",cols="6,2,2",options="header"]
diff --git
a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/if-you-know-kettle.adoc
b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/if-you-know-kettle.adoc
index 9bea13e..2760552 100644
---
a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/if-you-know-kettle.adoc
+++
b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/if-you-know-kettle.adoc
@@ -17,16 +17,17 @@ under the License.
[[IfYouKnowKettle]]
:imagesdir: ../assets/images
:page-pagination:
+:description: Hop wants to allow data engineers to be able to deliver high
quality work, deliver that work fast and integrated with bleeding edge
technology.
= If You Know Kettle (Pentaho Data Integration)
== Why Hop?
-With Hop, we want to allow data engineers to be able to deliver high quality
work, deliver that work fast and integrated with bleeding edge technology.
+Hop wants to allow data engineers to be able to deliver high quality work,
deliver that work fast and integrated with bleeding edge technology.
-We want Hop to be completely open source, and are eager to hear your feedback
on our https://chat.project-hop.org[chat] and just as eager to see your bug
tickets and feature request in https://jira.project-hop.org[our JIRA].
+We want Hop to be completely open source, and are eager to hear your feedback
on our https://chat.project-hop.org[chat] and just as eager to see your bug
tickets and feature request in
https://issues.apache.org/jira/projects/HOP/issues[JIRA].
-As an open source first project, we'll start the
https://www.apache.org/[Apache Software Foundation]
https://incubator.apache.org/[incubation] process soon, and aim to become and
ASF https://projects.apache.org/[top level project] as soon as possible.
+As an open source first project, we started the https://www.apache.org/[Apache
Software Foundation] https://incubator.apache.org/[incubation] process in
September 2020 and aim to become and ASF https://projects.apache.org/[top level
project] as soon as possible.
Check our https://hop.apache.org/docs/qa/[Q&A] for more information on why Hop
was created and what the project is all about.
diff --git
a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/import-kettle-projects.adoc
b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/import-kettle-projects.adoc
index f8ce305..f1ebbd3 100644
---
a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/import-kettle-projects.adoc
+++
b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/import-kettle-projects.adoc
@@ -18,12 +18,13 @@ under the License.
:imagesdir: ../assets/images
:page-pagination:
:page-pagination-no-next:
+:description: Compatibility with Kettle/PDI was never a goal for Apache Hop
(Incubating), but since a lot of organizations have invested vast amounts of
resources in Kettle/PDI project development, the Apache Hop (Incubating)
community provides a way to import Kettle/PDI code into Hop and convert the
imported code the the Hop ways of working.
= Import Kettle (PDI) Projects in Apache Hop (Incubating)
As stated in the https://hop.apache.org/docs/qa/[Q&A], Apache Hop (Incubating)
used Kettle (aka Pentaho Data Integration or PDI) as a starting point in late
2019. A lot has happened in the meantime on both Apache Hop (Incubating) and
Pentaho Data Integration.
-Compatibility with Kettle/PDI never was a goal for Apache Hop (Incubating),
but since a lot of organizations have invested vast amounts of resources in
Kettle/PDI project development, the Apache Hop (Incubating) community provides
a way to import Kettle/PDI code into Hop and convert the imported code the the
Hop ways of working.
+Compatibility with Kettle/PDI was never a goal for Apache Hop (Incubating),
but since a lot of organizations have invested vast amounts of resources in
Kettle/PDI project development, the Apache Hop (Incubating) community provides
a way to import Kettle/PDI code into Hop and convert the imported code the the
Hop ways of working.
== Imported Items
diff --git a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/index.adoc
b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/index.adoc
index 59c0d6a..0612c55 100644
--- a/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/index.adoc
+++ b/docs/hop-tech-manual/modules/ROOT/pages/hop-vs-kettle/index.adoc
@@ -14,6 +14,8 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Hop initially (late 2019) started as a fork of the Kettle
(Pentaho Data Integration). Hop and Kettle/PDI are independent projects, each
with their own roadmap and priorities. Given these different roadmaps,
architectural vision and development track, Hop and Kettle/PDI are incompatible.
+
[[HopAndKettle]]
:imagesdir: ../assets/images
:page-pagination:
diff --git
a/docs/hop-user-manual/modules/ROOT/assets/images/hop-gui/pipeline/failed-pipeline.png
b/docs/hop-user-manual/modules/ROOT/assets/images/hop-gui/pipeline/failed-pipeline.png
index f2de0bb..15245ca 100644
Binary files
a/docs/hop-user-manual/modules/ROOT/assets/images/hop-gui/pipeline/failed-pipeline.png
and
b/docs/hop-user-manual/modules/ROOT/assets/images/hop-gui/pipeline/failed-pipeline.png
differ
diff --git a/docs/hop-user-manual/modules/ROOT/pages/concepts.adoc
b/docs/hop-user-manual/modules/ROOT/pages/concepts.adoc
index dfe8f20..27ec033 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/concepts.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/concepts.adoc
@@ -16,6 +16,7 @@ under the License.
////
[[Concepts]]
:imagesdir: ../assets/images
+:description: Hop comes with a number of concepts: a variety of tools, a large
number of metadata types, projects and enviromments. At the core of literally
everything in Hop is metadata.
= Concepts
diff --git a/docs/hop-user-manual/modules/ROOT/pages/hop-usps.adoc
b/docs/hop-user-manual/modules/ROOT/pages/hop-usps.adoc
index 2b03b91..9f5368d 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/hop-usps.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/hop-usps.adoc
@@ -14,6 +14,8 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Hop differentiates itself from other platforms through an
absolute focus on metadata, a visual code editor, a kernel + plugins
architecture, portable run configurations, unit and integration testing and
life cycle management.
+
[[USPs]]
= Hop Unique Selling Propositions
diff --git a/docs/hop-user-manual/modules/ROOT/pages/index.adoc
b/docs/hop-user-manual/modules/ROOT/pages/index.adoc
index 6dbd9f3..85cf94a 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/index.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/index.adoc
@@ -16,6 +16,7 @@ under the License.
////
[[Index]]
:imagesdir: ../assets/images
+:description: As a relatively young platform, Hop covers a area of
functionalities that is both wide and deep. This page provides an overview of
topic that may be of interest to you.
= Hop User Manual
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/create-pipeline.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/create-pipeline.adoc
index dfbdcaa..8bca2c9 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/create-pipeline.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/create-pipeline.adoc
@@ -16,15 +16,19 @@ under the License.
////
[[CreatePipeline]]
:imagesdir: ../assets/images
+:description: Pipelines are the fundamental building blocks of your Hop
projects. Pipelines read from data sources and perform operations (joins,
lookups, filter and lots more) on the data before it is written out to a target
platform.
+
= Create a Pipeline
-TODO: this content is based on the xref:getting-started/index.adoc[Getting
Started Guide].
-Elaborate more.
+== How pipelines work
+
+Pipelines are the fundamental building blocks of your Hop projects.
+
+Pipelines perform the heavy lifting: they read data from a variety of sources,
perform a number of operations (combine, clean, enrich, transform etc) and
write the data out to some target platform. A pipeline executes all of these
operations in a _predefined order_ and in _parallel_.
-Pipelines are the fundamentals of your project.
-A pipeline contains operations executed in a predefined order.
-Pipelines consist of a chain of transforms that read, process or write data.
-The image below shows a simple pipeline that reads data from a table, adds a
message to the data and sends out an email.
+In the image below, a very simple pipeline reads data from a database, adds a
message to the data and sends out an email.
+All of these operations are executed in a predefined order (read from the
database, add the message, send the mail) and in parallel.
+The pipelines executes these transforms, Imagine our database table or query
contains thousands of rows. The pipeline will start reading results from the
query, pass those on to the 'Add message' transform. Once the message has been
added, we'll send a mail from the Mail transform. All of these will operate in
parallel, so the Mail transform will already be sending mails while the table
input is still reading records from the table or query.
image::hop-gui/pipeline/simple-pipeline.png[Hop - Simple
Pipeline,75%,align="left"]
@@ -46,7 +50,7 @@ Some transforms can read from or write to other transforms
conditionally to or f
The hop is unaware of it.
Hops can be disabled by clicking on them, or through right-click -> disable.
-== Creation
+== Create a pipeline
Create a new pipeline through the work item dialog.
You'll be presented with the dialog shown below.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/errorhandling.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/errorhandling.adoc
index 607f063..b821ca9 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/errorhandling.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/errorhandling.adoc
@@ -16,6 +16,7 @@ under the License.
////
[[PipelineErrorHandling]]
:imagesdir: ../assets/images
+:description: When a major failure occurs in one of the transforms, the
pipeline is notified and halts all active operations. This is fine in most
cases, pipeline failures are typically handled in the parent workflow. In some
cases however, there are errors you'll want to handle gracefully without
halting the entire pipeline.
= Pipeline Error Handling
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/hop-pipeline-editor.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/hop-pipeline-editor.adoc
index 12c785c..63070b8 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/hop-pipeline-editor.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/hop-pipeline-editor.adoc
@@ -16,9 +16,14 @@ under the License.
////
[[HopPipelineEditor]]
:imagesdir: ../assets/images
+:description: You'll perform a lot of operations on the pipeline canvas, but a
lot of additional functionality is available from the pipeline editor's main
toolbar
= Toolbar
+You learned to create a pipeline in xref:pipeline/create-pipeline.adoc[Create
a Pipeline]. You'll perform a lot of operations on the pipeline canvas, but a
lot of additional functionality is available from the pipeline editor's main
toolbar.
+
+There are two other important toolbars to work with
xref:pipeline/pipeline-unit-testing.adoc[unit tests] and
xref:projects/index.adoc[projects and environments]. Check the respective pages
to learn more about managing your projects and environments and to learn how to
write unit tests for your pipelines.
+
Let's walk through the top toolbar:
image::getting-started/getting-started-pipeline-toolbar.png[Hop - Pipeline
Toolbar,align="left"]
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/metadata-injection.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/metadata-injection.adoc
index c42494d..6b84010 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/metadata-injection.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/metadata-injection.adoc
@@ -16,10 +16,11 @@ under the License.
////
[[MetadataInjection]]
:imagesdir: ../assets/images
+:description: Metadata injection inserts data from various sources into a
template pipeline at runtime to reduce repetitive tasks.
= Metadata Injection
-Metadata injection inserts data from various sources into a pipeline at
runtime to reduce repetitive tasks.
+Metadata injection inserts data from various sources into a template pipeline
at runtime to reduce repetitive tasks.
For example, you might have a simple pipeline to load transaction data values
from a supplier, filter specific values, and output them to a file.
If you have more than one supplier, you would need to run this simple pipeline
for each supplier.
diff --git a/docs/hop-user-manual/modules/ROOT/pages/pipeline/partitioning.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/partitioning.adoc
index eddc807..fb88b64 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/partitioning.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/partitioning.adoc
@@ -15,6 +15,7 @@ specific language governing permissions and limitations
under the License.
////
:imagesdir: ../assets/images
+:description: Partitioning allows you to distribute all the data from a set
into distinct subsets according to the rule applied on a table or row, where
these subsets form a partition of the original set with no item replicated into
multiple groups.
= Partitioning
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipeline-unit-testing.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipeline-unit-testing.adoc
index 69647e9..3605c21 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipeline-unit-testing.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipeline-unit-testing.adoc
@@ -18,6 +18,7 @@ under the License.
:imagesdir: ../../assets/images
:openvar: ${
:closevar: }
+:description: Hop unit tests simulate inputs in the form of _Input data sets_
and validates output against _Golden data sets_. A unit test is a combination
of zero or more input sets and golden data sets along with a bunch of tweaks
you can do to the pipelines prior to testing.
= Pipeline Unit Testing
diff --git a/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipelines.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipelines.adoc
index 7ae704e..b8eca4f 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipelines.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/pipelines.adoc
@@ -16,9 +16,14 @@ under the License.
////
[[Pipelines]]
:imagesdir: ../assets/images
+:description: Pipelines, together with workflows, are the main building blocks
in Hop. Pipelines perform the heavy data lifting: in a pipeline, you read data
from one or more sources, perform a number of operations (joins, lookups,
filters and lots more) and finally write the processed data to one or more
target platforms.
= Pipelines
+Pipelines, together with workflows, are the main building blocks in Hop.
Pipelines perform the heavy data lifting: in a pipeline, you read data from one
or more sources, perform a number of operations (joins, lookups, filters and
lots more) and finally write the processed data to one or more target platforms.
+
+Pipelines are an extensive topic. Check the pages below to learn more about
working with pipelines:
+
* xref:pipeline/hop-pipeline-editor.adoc[Pipeline Editor]
* xref:pipeline/create-pipeline.adoc[Create a Pipeline]
* xref:pipeline/run-preview-debug-pipeline.adoc[Run, Preview and Debug a
Pipeline]
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/run-preview-debug-pipeline.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/run-preview-debug-pipeline.adoc
index 977cf7c..fb4ad48 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/run-preview-debug-pipeline.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/run-preview-debug-pipeline.adoc
@@ -16,6 +16,7 @@ under the License.
////
[[RunPreviewDebugPipeline]]
:imagesdir: ../assets/images
+:description: Hop Gui offers a lot of functionality to let data developers
run, preview and debug pipelines. Feedback is provided through the UI, logging
and run metrics.
= Run, Preview and Debug a Pipeline
@@ -61,7 +62,8 @@ The transforms also show a little table icon, giving you
access to a preview of
image::hop-gui/pipeline/preview-pipeline.png[Preview pipeline,75%,align="left"]
-When a pipeline has failed a red failed icon is displayed on the transform
containing an error.
+When a pipeline fails, a red triangle is shown in the upper right corner of
the failed transform.
+Hover your mouse over the red error triangle to take a quick peek at the error
message. The full stacktrace is available in the logs. Check
xref:pipeline/errorhandling.adoc[pipeline error handling] to learn about how
you can handle errors in your pipeline gracefully (which is not necessarily
what you want).
image::hop-gui/pipeline/failed-pipeline.png[Failed pipeline,75%,align="left"]
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/plugins/import/kettle-import.adoc
b/docs/hop-user-manual/modules/ROOT/pages/plugins/import/kettle-import.adoc
index daafe8c..63a5142 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/plugins/import/kettle-import.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/plugins/import/kettle-import.adoc
@@ -19,10 +19,11 @@ under the License.
:openvar: ${
:closevar: }
+:description: Hop is an independent platform that originates from the same
code base as Kettle (Pentaho Data Integration). Even though Hop and Kettle are
not compatible, you can upgrade your existing Kettle projects by simply
importing them into Hop.
= Kettle Import
-Whenever you want to import metadata from Kettle you can use this plugin.
+Whenever you want to import metadata from Kettle (Pentaho Data Integration)
you can use this plugin.
== Usage
diff --git a/docs/hop-user-manual/modules/ROOT/pages/protips/index.adoc
b/docs/hop-user-manual/modules/ROOT/pages/protips/index.adoc
index 5e9ed0d..9b33106 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/protips/index.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/protips/index.adoc
@@ -16,6 +16,7 @@ under the License.
////
[[ProTips]]
:imagesdir: ../assets/images
+:description: Hop is a large platform with a lot of options. These pro tips
consist of useful variables, keyboard shortcuts and mouse actions etc
= Pro Tips
diff --git a/docs/hop-user-manual/modules/ROOT/pages/samples/samples.adoc
b/docs/hop-user-manual/modules/ROOT/pages/samples/samples.adoc
index 04e03ef..53caf68 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/samples/samples.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/samples/samples.adoc
@@ -14,6 +14,7 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Hop comes with a set of samples for workflows, pipelines,
actions, transforms and other metadata objects. These samples are included in
your default Hop installation as the Samples project.
[[Samples]]
= Samples
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/technology/azure/index.adoc
b/docs/hop-user-manual/modules/ROOT/pages/technology/azure/index.adoc
index d2187aa..097af33 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/technology/azure/index.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/technology/azure/index.adoc
@@ -16,6 +16,7 @@ under the License.
////
:documentationPath: /technology/azure/
:language: en_US
+:description: Hop supports Azure in Apache VFS (Blob Storage) and Azure Event
Hubs listeners and writers.
= Azure
@@ -35,4 +36,4 @@ Hop supports Azure through a number of metadata types:
Apache xref:vfs.adoc[VFS Support] in Hop allows you to directly read from a
multitude of file systems and protocols, including Azure:
-* xref:vfs/azure-blob-storage-vfs.adoc[Azure Blog Storage]: read from and
write data directly in Azure Blob Storage
\ No newline at end of file
+* xref:vfs/azure-blob-storage-vfs.adoc[Azure Blob Storage]: read from and
write data directly in Azure Blob Storage
\ No newline at end of file
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/technology/cassandra/index.adoc
b/docs/hop-user-manual/modules/ROOT/pages/technology/cassandra/index.adoc
index 2f903d3..47d9da2 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/technology/cassandra/index.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/technology/cassandra/index.adoc
@@ -16,6 +16,7 @@ under the License.
////
:documentationPath: /technology/cassandra/
:language: en_US
+:description: Apache Hop supports Apache Cassandra through a Cassandra
connection metadata type, an Exec CQL action and transforms for Cassandra input
and output, and an SSTable output.
= Cassandra Technology
@@ -46,3 +47,4 @@ Hop supports Cassandra in the following metadata objects:
* xref:pipeline/transforms/cassandra-input.adoc[Cassandra Input]: Reads from a
Cassandra cluster through a CQL query.
* xref:pipeline/transforms/cassandra-output.adoc[Cassandra Output]: Write data
to a table in a Cassandra cluster.
+* xref:pipeline/transforms/sstable-output.adoc[SSTable Output]: Write data to
a filesystem directory as a Cassandra SSTable.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/technology/google/index.adoc
b/docs/hop-user-manual/modules/ROOT/pages/technology/google/index.adoc
index 29f075b..7193c23 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/technology/google/index.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/technology/google/index.adoc
@@ -16,6 +16,7 @@ under the License.
////
:documentationPath: /technology/google/
:language: en_US
+:description: The Google platform is supported in Hop through a number of
(Beam) transforms and Apache VFS support for Google Drive and Google Cloud
Storage. Additionally, a number of external transform plugins are available.
= Google Technology
@@ -26,6 +27,15 @@ We briefly touch upon them below.
== Pipeline Transforms
+* xref:pipeline/transforms/beambigqueryinput.adoc[BigQuery Input (Beam)]
+* xref:pipeline/transforms/beambigqueryoutput.adoc[BigQuery Output (Beam)]
+* xref:pipeline/transforms/beambigtableinput.adoc[Bigtable Input (Beam)]
+* xref:pipeline/transforms/beambigtableinput.adoc[Bigtable Output (Beam)]
+* xref:pipeline/transforms/beamgcppublisher.adoc[GCP Pub/Sub Publisher (Beam)]
+* xref:pipeline/transforms/beamgcpsubscriber.adoc[GCP Pub/Sub Subscriber
(Beam)]
+
+Check the xref:plugins/external-plugins.adoc[external plugins] for more
information on a number of additional Google plugins that can't be or won't be
included with Apache Hop (Google Sheets Input and Output, Google Analytics)
+
== VFS
Apache xref:vfs.adoc[VFS Support] in Hop allows you to directly read from a
multitude of file systems and protocols, including Google:
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/index.adoc
b/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/index.adoc
index 1d7766c..961047e 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/index.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/index.adoc
@@ -16,6 +16,7 @@ under the License.
////
:documentationPath: /technology/neo4j/
:language: en_US
+:description: Apache Hop has extensive support for the Neo4j graph database
through a dedicated connection metadata type, a graph model type and tens of
transforms and actions. Additionally, Hop logging data can be written to a
Neo4j database and can be consulted directly from the Neo4j logging perspective
in Hop Gui.
= Neo4j
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/working-with-neo4j-data.adoc
b/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/working-with-neo4j-data.adoc
index c6e0a09..09dd9b9 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/working-with-neo4j-data.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/technology/neo4j/working-with-neo4j-data.adoc
@@ -14,6 +14,8 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Apache Hop has extensive support for the Neo4j graph database,
including a library of samples that explain how to load data to and read from
your Neo4j graphs through Apache Hop.
+
= Working with Neo4j data
:toc:
diff --git a/docs/hop-user-manual/modules/ROOT/pages/technology/technology.adoc
b/docs/hop-user-manual/modules/ROOT/pages/technology/technology.adoc
index 7da841f..3109c6d 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/technology/technology.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/technology/technology.adoc
@@ -15,9 +15,11 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Hop integrates with tens if not hundreds of different
technologies. The Azure, Apache Cassandra, Google and Neo4j platforms are
supported at various points in Hop and are covered in their own section.
+
= Technology
-This section of the documentation contains technology-stack specific
information.
+This section contains technology-stack specific information.
The goal is to inform you about how Apache Hop offers support for the
technology in question.
It also provides starting points for you if you want to learn more about the
technology.
@@ -25,5 +27,5 @@ The technologies supported by Hop currently include:
* xref:technology/azure/index.adoc[Azure]
* xref:technology/cassandra/index.adoc[Apache Cassandra]
+* xref:technology/google/index.adoc[Google]
* xref:technology/neo4j/index.adoc[Neo4j]
-* xref:technology/google/index.adoc[Google]
\ No newline at end of file
diff --git a/docs/hop-user-manual/modules/ROOT/pages/variables.adoc
b/docs/hop-user-manual/modules/ROOT/pages/variables.adoc
index cbbda20..6ce72d3 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/variables.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/variables.adoc
@@ -18,6 +18,7 @@ under the License.
:imagesdir: ../assets/images
:openvar: ${
:closevar: }
+:description: Well-designed Hop solutions never use hard coded values. Hop
offers variables on the global, project and environment levels, and allows
workflows and parameters to provide parameters, to set and read variables in
runtime etc.
= Variables
diff --git a/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc
b/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc
index ace1882..a91fca6 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs.adoc
@@ -16,6 +16,7 @@ under the License.
////
[[VFS]]
:imagesdir: ../assets/images
+:description: Apache VFS, part of the Apache Commons project, allows Hop data
developers to access a large variety of file and data formats over a convenient
url syntax.
= Virtual File Systems
diff --git a/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc
b/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc
index b1a0c86..2c17e55 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/aws-s3-vfs.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /vfs/
:language: en_US
+:description: Apache Hop supports reading from and writing to AWS S3 buckets
from almost anywhere in the platform through Apache VFS
= AWS S3
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc
b/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc
index b42f836..fee6cb6 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/azure-blob-storage-vfs.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /vfs/
:language: en_US
+:description: Apache Hop supports reading from and writing to Azure Blob
Storage from almost anywhere in the platform through Apache VFS
= Azure Storage
diff --git a/docs/hop-user-manual/modules/ROOT/pages/vfs/dropbox-vfs.adoc
b/docs/hop-user-manual/modules/ROOT/pages/vfs/dropbox-vfs.adoc
index beb706e..1f10795 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/dropbox-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/dropbox-vfs.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /vfs/
:language: en_US
+:description: Apache Hop supports reading from and writing to Dropbox from
almost anywhere in the platform through Apache VFS
= Dropbox VFS
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc
b/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc
index 1b08720..8e91533 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/google-cloud-storage-vfs.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /vfs/
:language: en_US
+:description: Apache Hop supports reading from and writing to Google Cloud
Storage from almost anywhere in the platform through Apache VFS
= Google Cloud Storage VFS
diff --git a/docs/hop-user-manual/modules/ROOT/pages/vfs/google-drive-vfs.adoc
b/docs/hop-user-manual/modules/ROOT/pages/vfs/google-drive-vfs.adoc
index 3bc9cc6..fbaf27e 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/vfs/google-drive-vfs.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/vfs/google-drive-vfs.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /vfs/
:language: en_US
+:description: Apache Hop supports reading from and writing to Google Drive
from almost anywhere in the platform through Apache VFS
= Google Drive VFS
diff --git a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions.adoc
index 83237e7..911f4f0 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/actions.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/actions.adoc
@@ -14,10 +14,14 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
////
+:description: Actions are used the objects that are executed in a workflow.
They are logical operators that do not manipulate your data but create actions
that have to be performed around your pipelines.
+
[[actions-plugins]]
= Action Plugins
-Actions are used to create a workflow, they are logical operators that do not
manipulate your data but create actions that have to be performed around your
pipelines.
+Actions are used the objects that are executed in a workflow. They are logical
operators that do not manipulate your data but create actions that have to be
performed around your pipelines.
+
+Actions in a workflow typically do not process your data, nor do they pass on
data to the next action. When an action finishes running, it returns a true or
false error code that can be used to drive the workflows behavior through
true/false or unconditional (ignore exit code) workflows hops.
// tag::website-links[]
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/create-workflow.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/create-workflow.adoc
index 398979c..cef658a 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/create-workflow.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/create-workflow.adoc
@@ -16,10 +16,11 @@ under the License.
////
[[CreateWorkflow]]
:imagesdir: ../assets/images
+:description: Workflows are process that orchestrate a variety of actions
defined between a start and an end point.
= Create a Workflow
-Workflows are process that orchestrate a variety of actions defined between a
starting and endpoint.
+Workflows are processes that orchestrate a variety of actions defined between
a start and an end point.
These actions can be, but are not limited to:
* Executing other workflows
@@ -28,7 +29,7 @@ These actions can be, but are not limited to:
* Error messaging handling
* Notification handling
-A simple workflow is displayed below, the flow will check if a folder is
empty, if not a pipeline is executed.
+A simple workflow is displayed below. This workflow will check if a folder is
empty, if not a pipeline is executed.
image::hop-gui/workflow/simple-workflow.png[Hop-Simple
Workflow,75%,align="left"]
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/run-debug-workflow.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/run-debug-workflow.adoc
index f350e5a..c4ad753 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/run-debug-workflow.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/run-debug-workflow.adoc
@@ -16,6 +16,7 @@ under the License.
////
[[RunDebugWorkflow]]
:imagesdir: ../assets/images
+:description: Running a workflow requires a workflow run configuration and
(optionally) a number of parameters.
= Run and Debug a Workflow
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-local-workflow-engine.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-local-workflow-engine.adoc
index f0f1113..c90d17d 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-local-workflow-engine.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-local-workflow-engine.adoc
@@ -16,11 +16,13 @@ under the License.
////
[[LocalWorkflowEngine]]
:imagesdir: ../assets/images
+:description: The local run configuration runs workflows on your local
environment.
+
= Local Workflow Engine
== Hop local workflow engine
-The local run configuration runs workflows locally.
+The local run configuration runs workflows on your local environment.
=== Options
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-remote-workflow-engine.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-remote-workflow-engine.adoc
index bcd1f4b..fea7ce7 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-remote-workflow-engine.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/native-remote-workflow-engine.adoc
@@ -16,6 +16,8 @@ under the License.
////
[[RemoteWorkflowEngine]]
:imagesdir: ../assets/images
+:description: The remote run configuration runs workflows on a remote Hop
Server. This run configuration requires little configuration, but requires a
Hop Server and a Hop Server metadata definition.
+
= Remote Workflow Engine
== Hop Remote workflow engine
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/workflow-run-configurations.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/workflow-run-configurations.adoc
index 9eaff93..d021676 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/workflow-run-configurations.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflow-run-configurations/workflow-run-configurations.adoc
@@ -16,6 +16,8 @@ under the License.
////
[[WorkflowRunConfigurations]]
:imagesdir: ../assets/images
+:description: Run configurations decouple the design and execution phases of
Hop workflow development. A workflow is a definition of _how_ data is
processed, a run configurations defines _where_ the workflow is executed. Hop
supports a local and remote workflow runtime engines
+
= Workflow Run Configurations
== Description
diff --git a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflows.adoc
b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflows.adoc
index eb453b7..4e8cb13 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/workflow/workflows.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/workflow/workflows.adoc
@@ -16,9 +16,12 @@ under the License.
////
[[Workflows]]
:imagesdir: ../assets/images
+:description: Workflows are one of the core building blocks in Apache Hop.
Where pipelines do the heavy data lifting, workflows take care of the
orchestration work: prepare the environment, fetch remote files, perform error
handling and executing child workflows and pipelines.
= Workflows
+Workflows are one of the core building blocks in Apache Hop. Where pipelines
do the heavy data lifting, workflows take care of the orchestration work:
prepare the environment, fetch remote files, perform error handling and
executing child workflows and pipelines.
+
** xref:workflow/create-workflow.adoc[Create a Workflow]
** xref:workflow/run-debug-workflow.adoc[Run and Debug a Workflow]
**
xref:workflow/workflow-run-configurations/workflow-run-configurations.adoc[Workflow
Run Configurations]
