This is an automated email from the ASF dual-hosted git repository.
hansva pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hop.git
The following commit(s) were added to refs/heads/main by this push:
new c8669d77a8 Add the supported engines section to the transform pages
that had none, and drop Beam Direct. #8958 (#8961)
c8669d77a8 is described below
commit c8669d77a8659a8befc9217741b3b2c5f56a5d66
Author: Adalennis <[email protected]>
AuthorDate: Fri Oct 16 12:53:22 2026 +0200
Add the supported engines section to the transform pages that had none, and
drop Beam Direct. #8958 (#8961)
---
.../pages/pipeline/transforms/beamdebeziuminput.adoc | 2 +-
.../pipeline/transforms/beamelasticsearchinput.adoc | 2 +-
.../pipeline/transforms/beamelasticsearchoutput.adoc | 2 +-
.../ROOT/pages/pipeline/transforms/beammqttinput.adoc | 2 +-
.../ROOT/pages/pipeline/transforms/beammqttoutput.adoc | 2 +-
.../pages/pipeline/transforms/beamsnowflakeinput.adoc | 2 +-
.../pages/pipeline/transforms/beamsnowflakeoutput.adoc | 2 +-
.../pages/pipeline/transforms/beamsplunkoutput.adoc | 2 +-
.../ROOT/pages/pipeline/transforms/embedtext.adoc | 17 +----------------
.../ROOT/pages/pipeline/transforms/extractgraph.adoc | 17 +----------------
.../ROOT/pages/pipeline/transforms/jms-consumer.adoc | 1 +
.../ROOT/pages/pipeline/transforms/jms-producer.adoc | 1 +
.../pages/pipeline/transforms/pgvector-search.adoc | 18 +-----------------
.../pages/pipeline/transforms/pgvector-upsert.adoc | 18 +-----------------
.../ROOT/pages/pipeline/transforms/plugincatalog.adoc | 17 +----------------
.../ROOT/pages/pipeline/transforms/redis-input.adoc | 1 +
.../ROOT/pages/pipeline/transforms/redis-output.adoc | 1 +
.../pages/pipeline/transforms/structuredextract.adoc | 17 +----------------
.../ROOT/pages/pipeline/transforms/textchunker.adoc | 17 +----------------
19 files changed, 19 insertions(+), 122 deletions(-)
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamdebeziuminput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamdebeziuminput.adoc
index 4ea4f8c55a..c7f513c1c9 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamdebeziuminput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamdebeziuminput.adoc
@@ -17,7 +17,7 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Read database change events using Apache Beam DebeziumIO.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=no, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=no, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-debezium-input.svg[Beam Debezium input Icon,
role="image-doc-icon"] Beam Debezium input
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchinput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchinput.adoc
index 8db8a0dadd..77b7fb0c13 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchinput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchinput.adoc
@@ -17,7 +17,7 @@ limitations under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Beam Elasticsearch input uses the Beam ElasticsearchIO connector
for JSON documents.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=yes, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=yes, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-elasticsearch-input.svg[Beam Elasticsearch input
icon, role="image-doc-icon"] Beam Elasticsearch input
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchoutput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchoutput.adoc
index 31fbf980bb..9b4510cb24 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchoutput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamelasticsearchoutput.adoc
@@ -17,7 +17,7 @@ limitations under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Beam Elasticsearch output uses the Beam ElasticsearchIO
connector for JSON documents.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=yes, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=yes, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-elasticsearch-output.svg[Beam Elasticsearch
output icon, role="image-doc-icon"] Beam Elasticsearch output
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttinput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttinput.adoc
index 567e9e5e8d..58a293aa35 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttinput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttinput.adoc
@@ -17,7 +17,7 @@ limitations under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Beam MQTT input using Apache Beam MqttIO.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=yes, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=yes, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-mqtt-input.svg[Beam MQTT input icon,
role="image-doc-icon"] Beam MQTT input
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttoutput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttoutput.adoc
index 127f33d8df..fdc9beb00d 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttoutput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beammqttoutput.adoc
@@ -17,7 +17,7 @@ limitations under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Beam MQTT output using Apache Beam MqttIO.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=yes, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=yes, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-mqtt-output.svg[Beam MQTT output icon,
role="image-doc-icon"] Beam MQTT output
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeinput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeinput.adoc
index 69237ffa52..3cee05474d 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeinput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeinput.adoc
@@ -17,7 +17,7 @@ limitations under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Beam Snowflake input using a bounded GCS COPY.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=yes, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=yes, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-snowflake-input.svg[Beam Snowflake input icon,
role="image-doc-icon"] Beam Snowflake input
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeoutput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeoutput.adoc
index 87ffcee5cb..cd1b70784d 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeoutput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsnowflakeoutput.adoc
@@ -17,7 +17,7 @@ limitations under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Beam Snowflake output using a bounded GCS COPY.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=yes, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=yes, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-snowflake-output.svg[Beam Snowflake output icon,
role="image-doc-icon"] Beam Snowflake output
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsplunkoutput.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsplunkoutput.adoc
index bb2bce7f70..c24fcf4abb 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsplunkoutput.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/beamsplunkoutput.adoc
@@ -17,7 +17,7 @@ limitations under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Beam Splunk output using Apache Beam SplunkIO.
-:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Direct=yes, Beam Spark=yes, Beam Flink=yes, Beam Dataflow=yes
+:page-engines: Hop Engine=no, Single Threaded=no, Native Spark=no, Beam
Spark=yes, Beam Flink=yes, Beam Dataflow=yes
:page-marketplace: hop-engines-beam
= image:transforms/icons/beam-splunk-output.svg[Beam Splunk output icon,
role="image-doc-icon"] Beam Splunk output
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/embedtext.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/embedtext.adoc
index 0f36d147b8..0244d2af66 100644
--- a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/embedtext.adoc
+++ b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/embedtext.adoc
@@ -17,31 +17,16 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Turn a text field into an embedding vector using an AI provider.
+:page-engines: Hop Engine=yes, Single Threaded=yes, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
= image:transforms/icons/embedtext.svg[Embed text transform Icon,
role="image-doc-icon"] Embed text
-[%noheader,cols="3a,1a", role="table-no-borders" ]
-|===
-|
== Description
The Embed text transform turns a text field into an embedding vector by
calling the embedding model of an xref:metadata-types/ai-provider.adoc[AI
provider]. It is the middle step of a retrieval pipeline: chunk the documents,
embed the chunks, then write them to a vector store.
One row in, one row out. The embedding is added to the row, so nothing already
on it is lost.
-|
-== Supported Engines
-[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
-!===
-!Hop Engine! image:check_mark.svg[Supported, 24]
-!Single Threaded! image:check_mark.svg[Supported, 24]
-!Native Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Flink! image:question_mark.svg[Maybe Supported, 24]
-!Beam Dataflow! image:question_mark.svg[Maybe Supported, 24]
-!===
-|===
-
== Choosing the model
The model comes from the **EMBEDDING** entry of the selected AI provider, so
one provider can serve a chat transform and this transform at the same time,
with the credentials configured once.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/extractgraph.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/extractgraph.adoc
index 4cf6c85438..30e454aa9e 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/extractgraph.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/extractgraph.adoc
@@ -17,12 +17,10 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Read the entities in a text and the relationships between them
with an AI provider, for a knowledge graph.
+:page-engines: Hop Engine=yes, Single Threaded=yes, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
= image:transforms/icons/extractgraph.svg[Extract graph transform Icon,
role="image-doc-icon"] Extract graph
-[%noheader,cols="3a,1a", role="table-no-borders" ]
-|===
-|
== Description
The Extract graph transform reads the entities a text mentions and the
relationships between them, using the chat model of an
xref:metadata-types/ai-provider.adoc[AI provider].
@@ -34,19 +32,6 @@ A row without text, or a text without any entity or
relationship, produces nothi
This is the extraction step of GraphRAG: split documents with the
xref:pipeline/transforms/textchunker.adoc[Text chunker], extract the graph from
every chunk, and write the entities and relationships with
xref:pipeline/transforms/graph-output.adoc[Graph output], next to the chunks
and their xref:pipeline/transforms/embedtext.adoc[embeddings].
See xref:technology/graph-databases/index.adoc#vectors[Vectors and embeddings]
for storing and searching the embeddings in a graph database.
-|
-== Supported Engines
-[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
-!===
-!Hop Engine! image:check_mark.svg[Supported, 24]
-!Single Threaded! image:check_mark.svg[Supported, 24]
-!Native Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Flink! image:question_mark.svg[Maybe Supported, 24]
-!Beam Dataflow! image:question_mark.svg[Maybe Supported, 24]
-!===
-|===
-
== Output fields
[options="header"]
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-consumer.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-consumer.adoc
index a5c9d21691..784dfc5970 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-consumer.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-consumer.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: JMS consumer reads messages from a JMS queue or topic and turns
each into a row.
+:page-engines: Hop Engine=yes, Single Threaded=maybe, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
= image:transforms/icons/jms-consumer.svg[JMS consumer transform Icon,
role="image-doc-icon"] JMS consumer
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-producer.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-producer.adoc
index f5e494e27e..2a8b1d55f8 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-producer.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/jms-producer.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: JMS producer sends each incoming row to a JMS queue or topic as
a text message.
+:page-engines: Hop Engine=yes, Single Threaded=maybe, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
= image:transforms/icons/jms-producer.svg[JMS producer transform Icon,
role="image-doc-icon"] JMS producer
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-search.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-search.adoc
index de5eab2a9c..85235c3dec 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-search.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-search.adoc
@@ -17,13 +17,11 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Run a top-k similarity search against a PostgreSQL table that
uses the pgvector extension.
+:page-engines: Hop Engine=yes, Single Threaded=yes, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
:page-marketplace: hop-tech-pgvector
= image:transforms/icons/vector-store.svg[pgvector search transform Icon,
role="image-doc-icon"] pgvector search
-[%noheader,cols="3a,1a", role="table-no-borders" ]
-|===
-|
== Description
The pgvector search transform looks up the nearest stored chunks for a query
embedding in a PostgreSQL table that uses the
https://github.com/pgvector/pgvector[pgvector] extension. It is the read half
of a retrieval pipeline: embed the question, search, then feed the matches to a
language model.
@@ -32,20 +30,6 @@ Each incoming row carries one query embedding, and the
transform emits one outpu
It uses a regular xref:database/databases/postgresql.adoc[PostgreSQL]
relational database connection.
-
-|
-== Supported Engines
-[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
-!===
-!Hop Engine! image:check_mark.svg[Supported, 24]
-!Single Threaded! image:check_mark.svg[Supported, 24]
-!Native Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Flink! image:question_mark.svg[Maybe Supported, 24]
-!Beam Dataflow! image:question_mark.svg[Maybe Supported, 24]
-!===
-|===
-
== Embedding fields
As in pgvector upsert, the query embedding field may be either a field of the
**Vector** value type or a String field holding a JSON array.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-upsert.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-upsert.adoc
index e1292580bc..c93ef029a8 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-upsert.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/pgvector-upsert.adoc
@@ -17,13 +17,11 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Upsert text chunks and their embeddings into a PostgreSQL table
that uses the pgvector extension.
+:page-engines: Hop Engine=yes, Single Threaded=yes, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
:page-marketplace: hop-tech-pgvector
= image:transforms/icons/vector-store.svg[pgvector upsert transform Icon,
role="image-doc-icon"] pgvector upsert
-[%noheader,cols="3a,1a", role="table-no-borders" ]
-|===
-|
== Description
The pgvector upsert transform writes text chunks and their embedding vectors
into a PostgreSQL table that uses the
https://github.com/pgvector/pgvector[pgvector] extension. It is the write half
of a retrieval pipeline: typically Text Chunker, an embedding transform, then
this transform.
@@ -32,20 +30,6 @@ It uses a regular
xref:database/databases/postgresql.adoc[PostgreSQL] relational
The transform can create the extension, the table and an HNSW index on first
run, so an empty database is enough to get started.
-
-|
-== Supported Engines
-[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
-!===
-!Hop Engine! image:check_mark.svg[Supported, 24]
-!Single Threaded! image:check_mark.svg[Supported, 24]
-!Native Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Flink! image:question_mark.svg[Maybe Supported, 24]
-!Beam Dataflow! image:question_mark.svg[Maybe Supported, 24]
-!===
-|===
-
== Embedding fields
The embedding field may be either:
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/plugincatalog.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/plugincatalog.adoc
index 2e2c794a66..b7b9277b38 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/plugincatalog.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/plugincatalog.adoc
@@ -17,12 +17,10 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: The Plugin Catalog transform reads the live Hop plugin registry
and emits the installed transforms, actions and metadata types as rows.
+:page-engines: Hop Engine=yes, Single Threaded=yes, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
= image:transforms/icons/plugincatalog.svg[Plugin Catalog transform Icon,
role="image-doc-icon"] Plugin Catalog
-[%noheader,cols="3a,1a", role="table-no-borders" ]
-|===
-|
== Description
The Plugin Catalog transform reads the live Hop plugin registry and emits one
row per installed plugin: transforms, workflow actions and metadata types.
@@ -33,19 +31,6 @@ Because the catalog comes from the registry of the running
Hop installation rath
This is a source transform: it generates rows and does not read an input
stream.
-|
-== Supported Engines
-[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
-!===
-!Hop Engine! image:check_mark.svg[Supported, 24]
-!Single Threaded! image:check_mark.svg[Supported, 24]
-!Native Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Flink! image:question_mark.svg[Maybe Supported, 24]
-!Beam Dataflow! image:question_mark.svg[Maybe Supported, 24]
-!===
-|===
-
== Usage
The catalog answers questions about what a given Hop installation actually
contains, for example:
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-input.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-input.adoc
index 956163aad2..8ea6b2af5f 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-input.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-input.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Read STRING, HASH, SET or LIST values from Redis.
+:page-engines: Hop Engine=yes, Single Threaded=maybe, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
:page-marketplace: hop-tech-redis
= Redis Input
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-output.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-output.adoc
index 27da2b293f..7c829ce375 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-output.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/redis-output.adoc
@@ -17,6 +17,7 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Write STRING, HASH, SET or LIST values to Redis.
+:page-engines: Hop Engine=yes, Single Threaded=maybe, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
:page-marketplace: hop-tech-redis
= Redis Output
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/structuredextract.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/structuredextract.adoc
index 71604eb19f..3b5dbfa49c 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/structuredextract.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/structuredextract.adoc
@@ -17,12 +17,10 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Pull named, typed fields out of text using an AI provider.
+:page-engines: Hop Engine=yes, Single Threaded=yes, Native Spark=maybe, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
= image:transforms/icons/structuredextract.svg[Structured extract transform
Icon, role="image-doc-icon"] Structured extract
-[%noheader,cols="3a,1a", role="table-no-borders" ]
-|===
-|
== Description
The Structured extract transform reads named, typed fields out of a text field
using the model of an xref:metadata-types/ai-provider.adoc[AI provider]. A
contract becomes a renewal date and a value; a support ticket becomes a
severity and a product; an invoice becomes a supplier and a total.
@@ -31,19 +29,6 @@ One row in, one row out, with the extracted fields added to
it.
This is the step that turns text a pipeline could only carry into columns it
can filter, join and aggregate. It is not retrieval: nothing is searched and no
vector store is involved.
-|
-== Supported Engines
-[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
-!===
-!Hop Engine! image:check_mark.svg[Supported, 24]
-!Single Threaded! image:check_mark.svg[Supported, 24]
-!Native Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Flink! image:question_mark.svg[Maybe Supported, 24]
-!Beam Dataflow! image:question_mark.svg[Maybe Supported, 24]
-!===
-|===
-
== Fields to extract
The field grid is the whole configuration. Each row becomes three things at
once: a property in the schema the model is constrained by, a column added to
the stream, and the type the answer is read back into.
diff --git
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/textchunker.adoc
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/textchunker.adoc
index 92f0a0a85f..cb16bf1a64 100644
---
a/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/textchunker.adoc
+++
b/docs/hop-user-manual/modules/ROOT/pages/pipeline/transforms/textchunker.adoc
@@ -17,12 +17,10 @@ under the License.
:documentationPath: /pipeline/transforms/
:language: en_US
:description: Split text into chunks for embedding and retrieval, without
breaking words or sentences.
+:page-engines: Hop Engine=yes, Single Threaded=yes, Native Spark=yes, Beam
Spark=maybe, Beam Flink=maybe, Beam Dataflow=maybe
= image:transforms/icons/chunker.svg[Text chunker transform Icon,
role="image-doc-icon"] Text chunker
-[%noheader,cols="3a,1a", role="table-no-borders" ]
-|===
-|
== Description
The Text chunker transform splits a text field into smaller chunks and emits
one output row per chunk, carrying the fields of the input row along with each.
@@ -31,19 +29,6 @@ Chunking is the first step of a retrieval pipeline: an
embedding model has a bou
The transform has no dependency on an AI provider or a model. It is plain text
processing and can be used on its own wherever a large document needs to be
broken into passages.
-|
-== Supported Engines
-[%noheader,cols="2,1a",frame=none, role="table-supported-engines"]
-!===
-!Hop Engine! image:check_mark.svg[Supported, 24]
-!Single Threaded! image:check_mark.svg[Supported, 24]
-!Native Spark! image:check_mark.svg[Supported, 24]
-!Beam Spark! image:question_mark.svg[Maybe Supported, 24]
-!Beam Flink! image:question_mark.svg[Maybe Supported, 24]
-!Beam Dataflow! image:question_mark.svg[Maybe Supported, 24]
-!===
-|===
-
== Chunking strategies
[options="header"]
