This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a commit to branch dev
in repository https://gitbox.apache.org/repos/asf/seatunnel.git
The following commit(s) were added to refs/heads/dev by this push:
new b138a05da8 [Docs][Connector-V2] Improve HugeGraph Qdrant and Milvus
docs (#11329)
b138a05da8 is described below
commit b138a05da80d1ed85eeaa6e92630512b8abfb13c
Author: Daniel Carter <[email protected]>
AuthorDate: Sun Jul 5 16:14:57 2026 +0800
[Docs][Connector-V2] Improve HugeGraph Qdrant and Milvus docs (#11329)
Co-authored-by: DanielCarter-stack
<[email protected]>
---
docs/en/connectors/sink/HugeGraph.md | 34 ++++++++++++++++++++++++++++++++++
docs/en/connectors/sink/Milvus.md | 5 ++++-
docs/en/connectors/sink/Qdrant.md | 6 ++++--
docs/en/connectors/source/Milvus.md | 8 ++++++--
docs/zh/connectors/sink/HugeGraph.md | 34 ++++++++++++++++++++++++++++++++++
docs/zh/connectors/sink/Milvus.md | 5 ++++-
docs/zh/connectors/sink/Qdrant.md | 6 ++++--
docs/zh/connectors/source/Milvus.md | 8 ++++++--
8 files changed, 96 insertions(+), 10 deletions(-)
diff --git a/docs/en/connectors/sink/HugeGraph.md
b/docs/en/connectors/sink/HugeGraph.md
index fe545daaa6..9f24c73a0e 100644
--- a/docs/en/connectors/sink/HugeGraph.md
+++ b/docs/en/connectors/sink/HugeGraph.md
@@ -49,6 +49,8 @@ The HugeGraph schema must already exist before the job writes
data. Create the r
| `selected_fields` | List | No | - | A list of fields to
be selected from the input data. If not specified, all fields will be used.
|
| `ignored_fields` | List | No | - | A list of fields to
be ignored from the input data. Mutually exclusive with `selected_fields`.
|
+`selected_fields` and `ignored_fields` are applied before the row is mapped to
HugeGraph. Keep every field used by `idFields`, `sourceConfig.idFields`,
`targetConfig.idFields`, `mapping.fieldMapping`, or `mapping.sortKeys`;
otherwise the connector cannot build the vertex or edge ID.
+
### Schema Configuration (`schema_config`)
`schema_config` defines how one input stream is mapped to a specific vertex or
edge label in HugeGraph.
@@ -57,6 +59,7 @@ The HugeGraph schema must already exist before the job writes
data. Create the r
| ------------------ |--------------------| ---------- | -------------
|----------------------------------------------------------------------------------------------------------|
| `type` | String | Yes | - | The
type of graph element to map to. Must be `VERTEX` or `EDGE`.
|
| `label` | String | Yes | - | The
label of the vertex or edge in HugeGraph.
|
+| `tablePath` | String | No | - |
Reserved table path value carried in the schema config.
|
| `properties` | `List<String>` | No | - | A
list of source field names for the vertex or edge.
|
| `ttl` | Long | No | - | The
time-to-live for the vertex or edge in seconds.
|
| `ttlStartTime` | String | No | - | The
start time for the TTL.
|
@@ -92,6 +95,37 @@ This object provides advanced control over how fields and
values are mapped to p
| `timeZone` | String | No | `GMT+8` | The
time zone for date parsing.
|
| `sortKeys` | `List<String>` | For Edge | - | A
list of property keys to sort edges with the same source and target vertices.
|
+## Supported Types
+
+The connector validates the SeaTunnel row schema against the existing
HugeGraph schema before writing.
+
+| SeaTunnel type | HugeGraph property type |
+|----------------|-------------------------|
+| `BYTES` | `BLOB` |
+| `TINYINT` | `INT` |
+| `SMALLINT` | `INT` |
+| `INT` | `INT` |
+| `BIGINT` | `LONG` |
+| `FLOAT` | `FLOAT` |
+| `DOUBLE` | `DOUBLE` |
+| `BOOLEAN` | `BOOLEAN` |
+| `DATE` | `DATE` |
+| `TIMESTAMP` | `DATE` |
+| `ARRAY` | A non-single-cardinality HugeGraph property whose element
type is compatible |
+| `STRING` | `TEXT` |
+| `DECIMAL` | `TEXT` |
+| `MAP` | `TEXT` |
+| `ROW` | `TEXT` |
+| `TIME` | `TEXT` |
+| `NULL` | `TEXT` |
+
+## Write Behavior Notes
+
+- For vertices, `idStrategy` controls how the vertex ID is built.
`PRIMARY_KEY` joins all `idFields` with HugeGraph's primary-key format,
`CUSTOMIZE_STRING` joins them with `:`, `CUSTOMIZE_NUMBER` expects one numeric
field, and `CUSTOMIZE_UUID` expects one UUID field.
+- For edges, the connector reads the ID strategy from the existing source and
target vertex labels in HugeGraph. `sourceConfig.idFields` and
`targetConfig.idFields` must provide enough fields to rebuild those vertex IDs.
+- `INSERT` writes a new vertex or edge, `UPDATE_AFTER` updates the existing
graph element, and `DELETE` deletes it. Delete rows only need the fields
required to build the element ID.
+- `mapping.nullValues` treats matching string values as null and skips those
properties during writes.
+
## Usage Examples
The examples below assume the corresponding HugeGraph schema has already been
created.
diff --git a/docs/en/connectors/sink/Milvus.md
b/docs/en/connectors/sink/Milvus.md
index 6dd7b794c2..30ba9c072c 100644
--- a/docs/en/connectors/sink/Milvus.md
+++ b/docs/en/connectors/sink/Milvus.md
@@ -53,7 +53,7 @@ Common use cases:
| url | String | Yes | -
| The URL to connect to Milvus or Zilliz Cloud, for example
`http://127.0.0.1:19530`.
|
| token | String | Yes | -
| Milvus authentication token. For a local Milvus server this is
usually `username:password`.
|
| database | String | No | -
| Target database. If it is not set, the sink uses the upstream
database when available.
|
-| collection | String | No | -
| Target collection. If it is not set, the sink uses the upstream
table name. The deprecated `collection_name` key is accepted as an alias.
|
+| collection | String | No | -
| Target collection. If it is not set, the sink uses the upstream
table name. The legacy key `collection_name` is still accepted as a fallback
alias.
|
| schema_save_mode | enum | No |
CREATE_SCHEMA_WHEN_NOT_EXIST | Controls how SeaTunnel handles the target
collection schema. The default creates the collection only when it does not
already exist.
|
| data_save_mode | enum | No | APPEND_DATA
| Controls how SeaTunnel handles existing data. Supported values are
`DROP_DATA`, `APPEND_DATA`, and `ERROR_WHEN_DATA_EXISTS`.
|
| enable_auto_id | boolean | No | false
| Whether Milvus should generate primary key values automatically. If
this is `true`, do not send values for the primary key field.
|
@@ -70,8 +70,11 @@ Common use cases:
- Vector dimensions come from the SeaTunnel schema. For vector fields
generated by `FakeSource`, `columnScale` is used as the vector dimension.
- If `collection` is not set, the sink uses the upstream table name as the
Milvus collection name. This is useful for multi-table jobs.
+- `collection_name` is a backward-compatible alias for `collection`. Prefer
`collection` in new jobs.
- When a Milvus source reads partitions, the sink can create the same
partition names on the target collection if the target collection does not use
a partition key.
- `create_index = true` only creates vector indexes when index metadata is
available from the upstream catalog, such as when the source is Milvus.
+- `enable_upsert = true` writes by primary key and requires a primary key in
the target collection. Use `enable_upsert = false` when writing rows that
should be inserted directly.
+- `load_collection = true` loads the collection after creation so it can be
queried immediately after the write finishes.
## Task Example
diff --git a/docs/en/connectors/sink/Qdrant.md
b/docs/en/connectors/sink/Qdrant.md
index d73297b36a..0ab9fa6beb 100644
--- a/docs/en/connectors/sink/Qdrant.md
+++ b/docs/en/connectors/sink/Qdrant.md
@@ -21,7 +21,7 @@ The Qdrant sink writes SeaTunnel rows into one existing
Qdrant collection. Norma
| name | type | required | default value | description |
|-----------------|--------|----------|---------------|-------------|
-| collection_name | string | yes | - | Qdrant collection name
to write to. |
+| collection_name | string | yes | - | Qdrant collection name
to write to. Set this explicitly for normal jobs. |
| host | string | no | localhost | Qdrant gRPC host. |
| port | int | no | 6334 | Qdrant gRPC port. |
| api_key | string | no | "" | Qdrant API key for
authenticated deployments. |
@@ -30,7 +30,7 @@ The Qdrant sink writes SeaTunnel rows into one existing
Qdrant collection. Norma
### collection_name [string]
-The name of the Qdrant collection to write.
+The name of the Qdrant collection to write. The target collection must already
exist and its vector names and dimensions must match the SeaTunnel vector
columns.
### host [string]
@@ -75,7 +75,9 @@ The value of the primary key column is used as the Qdrant
point ID. Primary key
- The target collection must already exist before the job starts. The
connector does not create collections or vector indexes.
- Vector column names and dimensions must match the vector configuration of
the target Qdrant collection.
+- The sink writes vector columns as named vectors. Use a target collection
with named vectors that match the SeaTunnel vector column names.
- The sink writes each incoming row as an upsert request. It does not
interpret `UPDATE` or `DELETE` row kinds as CDC operations.
+- The writer buffers up to 64 points and also flushes when the writer is
closed or a commit is prepared. This batch size is not currently exposed as a
connector option.
- Each sink block writes to one collection. Use separate sink blocks when
different collections need different settings.
## Task Example
diff --git a/docs/en/connectors/source/Milvus.md
b/docs/en/connectors/source/Milvus.md
index d90bbcc773..590ff0eb06 100644
--- a/docs/en/connectors/source/Milvus.md
+++ b/docs/en/connectors/source/Milvus.md
@@ -22,7 +22,9 @@ Common use cases:
- [x] [batch](../../introduction/concepts/connector-v2-features.md)
- [x] [exactly-once](../../introduction/concepts/connector-v2-features.md)
-- [ ] [column projection](../../introduction/concepts/connector-v2-features.md)
+- [x] [column projection](../../introduction/concepts/connector-v2-features.md)
+- [x] [parallelism](../../introduction/concepts/connector-v2-features.md)
+- [x] [support multiple table
read](../../introduction/concepts/connector-v2-features.md)
## Data Type Mapping
@@ -51,7 +53,7 @@ Common use cases:
| url | String | Yes | - | The URL to connect to Milvus or
Zilliz Cloud, for example `http://127.0.0.1:19530`.
|
| token | String | Yes | - | Milvus authentication token. For
a local Milvus server this is usually `username:password`.
|
| database | String | No | default | Source database.
|
-| collection | String | No | - | Source collection. If it is set,
only this collection is read. If it is not set, all collections under
`database` are read. The deprecated `collection_name` key is accepted as an
alias. |
+| collection | String | No | - | Source collection. If it is set,
only this collection is read. If it is not set, all collections under
`database` are read. |
| batch_size | Int | No | 1000 | Number of records fetched in one
source read batch.
|
| rate_limit | Int | No | 1000000 | Maximum source read rate limit.
|
@@ -59,6 +61,8 @@ Common use cases:
- `database` defaults to `default`, so simple local Milvus jobs do not need to
set it.
- `collection` is optional. Set it when the job should read exactly one
collection.
+- When `collection` is not set, the source discovers all collections in
`database` and exposes each collection as a separate SeaTunnel table.
+- The source splits work by Milvus partition. Collections with a partition key
are read with one split; collections without a partition key are split by
partition name and assigned across readers.
- When the source reads a collection with partitions, downstream Milvus sink
can use that metadata to create the same partition names on the target
collection.
- When the source reads vector indexes, downstream Milvus sink can use that
metadata with `create_index = true` to create matching vector indexes.
diff --git a/docs/zh/connectors/sink/HugeGraph.md
b/docs/zh/connectors/sink/HugeGraph.md
index ec6da2a7c8..203f50b622 100644
--- a/docs/zh/connectors/sink/HugeGraph.md
+++ b/docs/zh/connectors/sink/HugeGraph.md
@@ -49,6 +49,8 @@ HugeGraph sink连接器允许您将数据从SeaTunnel写入Apache HugeGraph，
| `selected_fields` | List | 否 | - |
要从输入数据中选择的字段列表。如果未指定，将使用所有字段。 |
| `ignored_fields` | List | 否 | - |
要从输入数据中忽略的字段列表。与`selected_fields`互斥。 |
+`selected_fields` 和 `ignored_fields` 会在字段映射到 HugeGraph 之前生效。请保留
`idFields`、`sourceConfig.idFields`、`targetConfig.idFields`、`mapping.fieldMapping`
或 `mapping.sortKeys` 会用到的字段，否则连接器无法生成顶点或边的 ID。
+
### Schema配置 (`schema_config`)
`schema_config` 定义一个输入流如何映射到 HugeGraph 中的某个顶点标签或边标签。
@@ -57,6 +59,7 @@ HugeGraph sink连接器允许您将数据从SeaTunnel写入Apache HugeGraph，
| ------------------ | ------------------- | -------- | -------
|------------------------------------------------------------|
| `type` | String | 是 | - |
要映射到的图元素的类型。必须是`VERTEX`或`EDGE`。 |
| `label` | String | 是 | - |
HugeGraph中顶点或边的标签。 |
+| `tablePath` | String | 否 | - | Schema
配置中携带的预留表路径值。 |
| `properties` | `List<String>` | 否 | - |
顶点或边的源字段名称列表。 |
| `ttl` | Long | 否 | - | 顶点或边的生存时间（秒）。
|
| `ttlStartTime` | String | 否 | - | TTL的开始时间。
|
@@ -92,6 +95,37 @@ HugeGraph sink连接器允许您将数据从SeaTunnel写入Apache HugeGraph，
| `timeZone` | String | 否 | `GMT+8` | 用于日期解析的时区。
|
| `sortKeys` | `List<String>` | 对于边 | - |
用于对具有相同源和目标顶点的边进行排序的属性键列表。
|
+## 支持的数据类型
+
+写入前，连接器会校验 SeaTunnel 行结构和 HugeGraph 中已经存在的 Schema 是否匹配。
+
+| SeaTunnel 类型 | HugeGraph 属性类型 |
+|----------------|--------------------|
+| `BYTES` | `BLOB` |
+| `TINYINT` | `INT` |
+| `SMALLINT` | `INT` |
+| `INT` | `INT` |
+| `BIGINT` | `LONG` |
+| `FLOAT` | `FLOAT` |
+| `DOUBLE` | `DOUBLE` |
+| `BOOLEAN` | `BOOLEAN` |
+| `DATE` | `DATE` |
+| `TIMESTAMP` | `DATE` |
+| `ARRAY` | HugeGraph 中非单值属性，且数组元素类型兼容 |
+| `STRING` | `TEXT` |
+| `DECIMAL` | `TEXT` |
+| `MAP` | `TEXT` |
+| `ROW` | `TEXT` |
+| `TIME` | `TEXT` |
+| `NULL` | `TEXT` |
+
+## 写入行为说明
+
+- 写入顶点时，`idStrategy` 决定如何生成顶点 ID。`PRIMARY_KEY` 会按 HugeGraph 主键格式拼接所有
`idFields`，`CUSTOMIZE_STRING` 会用 `:` 拼接字段，`CUSTOMIZE_NUMBER`
需要一个数字字段，`CUSTOMIZE_UUID` 需要一个 UUID 字段。
+- 写入边时，连接器会从 HugeGraph 中已有的源顶点标签和目标顶点标签读取 ID 策略。`sourceConfig.idFields` 和
`targetConfig.idFields` 必须能还原对应顶点 ID。
+- `INSERT` 会写入新的顶点或边，`UPDATE_AFTER` 会更新已有图元素，`DELETE` 会删除图元素。删除行只需要包含能生成图元素 ID
的字段。
+- `mapping.nullValues` 中列出的字符串会被当作空值处理，写入时会跳过这些属性。
+
## 使用示例
下面示例默认 HugeGraph 中已经提前创建好对应 Schema。
diff --git a/docs/zh/connectors/sink/Milvus.md
b/docs/zh/connectors/sink/Milvus.md
index 1dad0edf93..9dbd15d962 100644
--- a/docs/zh/connectors/sink/Milvus.md
+++ b/docs/zh/connectors/sink/Milvus.md
@@ -53,7 +53,7 @@ SeaTunnel 表结构创建目标集合，并支持向量字段、动态字段、
| url | String | 是 | -
| Milvus 或 Zilliz Cloud 的连接地址，例如 `http://127.0.0.1:19530`。
|
| token | String | 是 | -
| Milvus 认证令牌。本地 Milvus 通常使用 `username:password`。
|
| database | String | 否 | -
| 目标数据库。不填时，如果上游带有数据库信息，则使用上游数据库。
|
-| collection | String | 否 | -
| 目标集合。不填时使用上游表名作为集合名。兼容旧配置键 `collection_name`。
|
+| collection | String | 否 | -
| 目标集合。不填时使用上游表名作为集合名。旧配置键 `collection_name` 仍可作为兼容别名使用。
|
| schema_save_mode | enum | 否 |
CREATE_SCHEMA_WHEN_NOT_EXIST | 控制如何处理目标集合结构。默认值表示集合不存在时才创建集合。
|
| data_save_mode | enum | 否 | APPEND_DATA
| 控制如何处理已有数据。支持 `DROP_DATA`、`APPEND_DATA`、`ERROR_WHEN_DATA_EXISTS`。
|
| enable_auto_id | boolean | 否 | false
| 是否让 Milvus 自动生成主键。设置为 `true` 时，不要再向主键字段写值。
|
@@ -70,8 +70,11 @@ SeaTunnel 表结构创建目标集合，并支持向量字段、动态字段、
- 向量维度来自 SeaTunnel 表结构。使用 `FakeSource` 生成向量字段时，`columnScale` 表示向量维度。
- 不配置 `collection` 时，接收器会使用上游表名作为 Milvus 集合名，适合多表写入。
+- `collection_name` 是 `collection` 的向后兼容别名。新任务建议使用 `collection`。
- Milvus 源端读取分区后，如果目标集合没有使用分区键，接收器可以在目标集合创建相同分区名。
- `create_index = true` 只有在上游目录信息能提供索引元数据时才会创建向量索引，例如上游也是 Milvus。
+- `enable_upsert = true` 会按主键更新插入，目标集合中需要有主键。希望直接追加写入时，可以设置 `enable_upsert =
false`。
+- `load_collection = true` 会在创建集合后加载集合，这样写入完成后可以马上查询。
## 任务示例
diff --git a/docs/zh/connectors/sink/Qdrant.md
b/docs/zh/connectors/sink/Qdrant.md
index 2f02c14212..fa9e947967 100644
--- a/docs/zh/connectors/sink/Qdrant.md
+++ b/docs/zh/connectors/sink/Qdrant.md
@@ -21,7 +21,7 @@ Qdrant sink 会把 SeaTunnel 行写入一个已经存在的 Qdrant collection。
| 名称 | 类型 | 必填 | 默认值 | 说明 |
|-----------------|--------|------|-----------|------|
-| collection_name | string | 是 | - | 要写入的 Qdrant collection 名称。 |
+| collection_name | string | 是 | - | 要写入的 Qdrant collection
名称。普通任务中请显式配置该项。 |
| host | string | 否 | localhost | Qdrant gRPC 主机。 |
| port | int | 否 | 6334 | Qdrant gRPC 端口。 |
| api_key | string | 否 | "" | 认证场景下使用的 Qdrant API key。 |
@@ -30,7 +30,7 @@ Qdrant sink 会把 SeaTunnel 行写入一个已经存在的 Qdrant collection。
### collection_name [string]
-要写入的 Qdrant collection 名称。
+要写入的 Qdrant collection 名称。目标 collection 必须已经存在，并且向量名、向量维度要和 SeaTunnel 向量列一致。
### host [string]
@@ -75,7 +75,9 @@ Sink 插件通用参数，请参考 [Sink 通用选项](../common-options/sink-c
- 目标 collection 必须在作业启动前已经存在。连接器不会自动创建 collection 或向量索引。
- 向量列名和维度必须与目标 Qdrant collection 的向量配置一致。
+- sink 会把向量列写成命名向量。目标 collection 中的命名向量需要和 SeaTunnel 向量列名一致。
- sink 会把每条输入行作为 upsert 写入，不会按 `UPDATE` 或 `DELETE` 行类型执行 CDC 语义。
+- 写入器最多缓存 64 个 point，并且在关闭写入器或准备提交时也会刷新。这一批次大小目前不是可配置项。
- 每个 sink 配置块写入一个 collection。如果不同 collection 需要不同配置，请使用多个 sink 配置块。
## 任务示例
diff --git a/docs/zh/connectors/source/Milvus.md
b/docs/zh/connectors/source/Milvus.md
index 2c6e2bf39c..c2543cdf8f 100644
--- a/docs/zh/connectors/source/Milvus.md
+++ b/docs/zh/connectors/source/Milvus.md
@@ -22,7 +22,9 @@ Milvus 源连接器用于从 Milvus 或 Zilliz Cloud 读取数据。它可以读
- [x] [批处理](../../introduction/concepts/connector-v2-features.md)
- [x] [精确一次](../../introduction/concepts/connector-v2-features.md)
-- [ ] [列投影](../../introduction/concepts/connector-v2-features.md)
+- [x] [列投影](../../introduction/concepts/connector-v2-features.md)
+- [x] [并行读取](../../introduction/concepts/connector-v2-features.md)
+- [x] [支持多表读取](../../introduction/concepts/connector-v2-features.md)
## 数据类型映射
@@ -51,7 +53,7 @@ Milvus 源连接器用于从 Milvus 或 Zilliz Cloud 读取数据。它可以读
| url | String | 是 | - | Milvus 或 Zilliz Cloud 的连接地址，例如
`http://127.0.0.1:19530`。 |
| token | String | 是 | - | Milvus 认证令牌。本地 Milvus 通常使用
`username:password`。 |
| database | String | 否 | default | 源数据库。
|
-| collection | String | 否 | - | 源集合。配置后只读取这个集合；不配置时读取 `database`
下的所有集合。兼容旧配置键 `collection_name`。 |
+| collection | String | 否 | - | 源集合。配置后只读取这个集合；不配置时读取 `database`
下的所有集合。 |
| batch_size | Int | 否 | 1000 | 每批从 Milvus 读取的数据条数。
|
| rate_limit | Int | 否 | 1000000 | 源端读取速率限制。
|
@@ -59,6 +61,8 @@ Milvus 源连接器用于从 Milvus 或 Zilliz Cloud 读取数据。它可以读
- `database` 默认是 `default`，本地 Milvus 的简单任务通常不用配置。
- `collection` 是可选项。只想读一个集合时再配置。
+- 不配置 `collection` 时，源端会发现 `database` 下的所有集合，并把每个集合作为一张独立的 SeaTunnel 表输出。
+- 源端会按 Milvus 分区拆分读取任务。有分区键的集合使用一个 split 读取；没有分区键的集合会按分区名拆分，并分配给多个 reader。
- 源端读取带分区的集合时，下游 Milvus 接收器可以利用这些元数据在目标集合创建相同分区名。
- 源端读取到向量索引信息时，下游 Milvus 接收器可以配合 `create_index = true` 创建相同向量索引。
