bitflicker64 commented on code in PR #464: URL: https://github.com/apache/hugegraph-doc/pull/464#discussion_r3910147829
########## PR_DESCRIPTION.md: ########## @@ -0,0 +1,31 @@ +# Add SeaTunnel Connector Integration Guide (CN only) Review Comment: ⚠️ important - This is a pull request description committed as a file in the site repository. The only Markdown files at the repository root on master are `AGENTS.md`, `README.md` and `contribution.md`; nothing references this file and `.gitignore` does not cover it, so it would ship as a permanent stray file. The same information already belongs in the PR description field. It also begins with a UTF-8 BOM: `git show 8956f3b:PR_DESCRIPTION.md | head -c 6 | xxd` returns `efbb bf23 2041`. Please delete `PR_DESCRIPTION.md` from the branch. ########## content/cn/docs/introduction/_index.md: ########## @@ -23,18 +23,9 @@ HugeGraph 支持百亿以上的顶点和边的快速存储与查询，具备出 ### 生态系统全景 -``` -┌──────────────────────────────────────────────────────────────┐ -│ Apache HugeGraph - Full-Stack Graph System │ -├──────────────────┬────────────────────┬──────────────────────┤ -│ Graph DB (OLTP) │ Graph Compute │ Graph AI │ -│ HugeGraph │ Vermeer (Memory) │ HugeGraph-AI │ -│ Server │ Computer (Dist.) │ GraphRAG/GNN/Py │ -├──────────────────┴────────────────────┴──────────────────────┤ -│ HugeGraph Toolchain │ -│ Hubble | Loader | Client(Java/Go/Py) | Spark | Tools │ -└──────────────────────────────────────────────────────────────┘ -``` + Review Comment: ⚠️ important - Removing the ASCII block is listed as intentional in the PR description, but the heading above is still `### 生态系统全景` and the figure that replaces it does not cover that subject. I opened `seatunnel-overview.png` at this head: it shows JDBC / Kafka / 文件 boxes, then SeaTunnel, then HugeGraph, with 读取 / 转换 / 写入 underneath. None of what the removed block listed survives: Graph DB (OLTP) HugeGraph Server, Graph Compute (Vermeer, Computer), Graph AI (HugeGraph-AI, GraphRAG/GNN/Py), and the Toolchain row (Hubble, Loader, Client, Spark, Tools). So this section loses content unrelated to SeaTunnel, and `content/en/docs/introduction/_index.md` still carries the block at lines 23-34, leaving the two language versions describing different things under the same heading. Please keep an ecosystem figure here, replacing the ASCII block with a diagram of the same scope if the alignment problems are the reason for dropping it. The SeaTunnel figure already appears on the new connector page at line 12, which is the right place for it. ########## content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md: ########## @@ -0,0 +1,410 @@ +--- +title: "HugeGraph-SeaTunnel Connector Quick Start" +linkTitle: "使用 SeaTunnel Connector 同步数据" +weight: 5 +--- + +SeaTunnel 负责连接数据源和数据目的地。HugeGraph Connector 提供两种能力。 + +- `HugeGraph Sink` 把文件、数据库、Kafka 等数据写入 HugeGraph。 +- `HugeGraph Source` 从 HugeGraph 读出顶点和边，已合入 `dev`（将随 SeaTunnel 3.0.0 发布），尚未随正式版本发布。 + + + +## 1 先看版本 + +本文把发布版和开发版分开写。配置放错版本，任务会在启动阶段失败。 + +| 使用内容 | SeaTunnel 版本 | 配置方式 | 状态 | +| --- | --- | --- | --- | +| JDBC / Kafka 写入 HugeGraph | 2.3.13 | `schema_config` | 发布版 | +| HugeGraph 读取和图迁移 | 当前 `dev` | HugeGraph Source + `mappings` | 已合入 dev，随 3.0.0 发布 | +| `graph2graph` | 当前 `dev` | Source + Sink | 已合入 dev，随 3.0.0 发布 | + +HugeGraph Source 与 `mappings` 多映射重构已随 PR [apache/seatunnel#11413](https://github.com/apache/seatunnel/pull/11413) 合入 dev（2026-08），将随 SeaTunnel **3.0.0** 正式发布。2.3.13 发行包内置 HugeGraph Client 1.5.0，dev 已升级到 1.7.0，需与 Server 版本匹配。 + +当前 `dev` 示例按 commit [`f1a1a0a`](https://github.com/apache/seatunnel/commit/f1a1a0abbe24bdac8cf23307995a78f778a3f467) 核对，固定版本的 [HugeGraph Source 文档](https://github.com/apache/seatunnel/blob/f1a1a0abbe24bdac8cf23307995a78f778a3f467/docs/zh/connectors/source/HugeGraph.md) 和 [HugeGraph Sink 文档](https://github.com/apache/seatunnel/blob/f1a1a0abbe24bdac8cf23307995a78f778a3f467/docs/zh/connectors/sink/HugeGraph.md) 与本文对应。`dev` 会继续变化，使用新版本前请重新核对配置。 + +2.3.13 的 HugeGraph Sink 使用 `schema_config`，这个版本没有 HugeGraph Source。本文所有发布版示例都按这个边界编写。 + +## 2 选哪个工具 + +先看数据从哪里来，以及任务是否已经属于一条大数据管道。 + +| 你的任务 | 推荐工具 | 适合原因 | +| --- | --- | --- | +| 管理图、执行 Gremlin、备份恢复、图克隆 | [HugeGraph-Tools](/cn/docs/quickstart/toolchain/hugegraph-tools) | 只操作 HugeGraph，命令直接 | +| 把本地文件、HDFS、MySQL 等数据批量导入 HugeGraph | [HugeGraph-Loader](/cn/docs/quickstart/toolchain/hugegraph-loader) | 配置简单，导入流程短 | +| 数据要经过 Kafka、JDBC、Flink、Spark 或多个外部系统 | SeaTunnel | 可以复用已有数据管道 | +| 需要流式任务、checkpoint 或统一管理多个连接器 | SeaTunnel | 支持 Source、Transform 和 Sink 组合 | +| 稳定地把一张 HugeGraph 图复制到另一张图 | Tools 优先 | 发布版工具更直接；SeaTunnel Source 仍是 dev 预览 | + +只维护一张图、没有现成大数据管道时，优先从 Loader 或 Tools 开始。SeaTunnel 需要额外准备连接器插件，并使用 HOCON 配置文件。 + +## 3 准备工作 + +### 3.1 HugeGraph + +本文示例使用以下图模型。 + +| 图元素 | 配置 | +| --- | --- | +| VertexLabel | `person`，主键为 `name` | +| PropertyKey | `name` 为 Text，`age` 为 Int | Review Comment: ⚠️ important - The PropertyKey row lists only `name` (Text) and `age` (Int), but the next row declares EdgeLabel `knows` with property `since`. HugeGraph rejects an EdgeLabel that references a PropertyKey which does not exist, so a reader who creates the schema exactly as this table states cannot create `knows`. Section 4.2's source table declares `since INT NOT NULL` and section 6.2's Source schema declares `since = "int"`, so the examples do expect it. Please add `since` as an Int PropertyKey to this row. ########## content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md: ########## @@ -0,0 +1,410 @@ +--- +title: "HugeGraph-SeaTunnel Connector Quick Start" +linkTitle: "使用 SeaTunnel Connector 同步数据" +weight: 5 +--- + +SeaTunnel 负责连接数据源和数据目的地。HugeGraph Connector 提供两种能力。 + +- `HugeGraph Sink` 把文件、数据库、Kafka 等数据写入 HugeGraph。 +- `HugeGraph Source` 从 HugeGraph 读出顶点和边，已合入 `dev`（将随 SeaTunnel 3.0.0 发布），尚未随正式版本发布。 + + + +## 1 先看版本 + +本文把发布版和开发版分开写。配置放错版本，任务会在启动阶段失败。 + +| 使用内容 | SeaTunnel 版本 | 配置方式 | 状态 | +| --- | --- | --- | --- | +| JDBC / Kafka 写入 HugeGraph | 2.3.13 | `schema_config` | 发布版 | +| HugeGraph 读取和图迁移 | 当前 `dev` | HugeGraph Source + `mappings` | 已合入 dev，随 3.0.0 发布 | +| `graph2graph` | 当前 `dev` | Source + Sink | 已合入 dev，随 3.0.0 发布 | + +HugeGraph Source 与 `mappings` 多映射重构已随 PR [apache/seatunnel#11413](https://github.com/apache/seatunnel/pull/11413) 合入 dev（2026-08），将随 SeaTunnel **3.0.0** 正式发布。2.3.13 发行包内置 HugeGraph Client 1.5.0，dev 已升级到 1.7.0，需与 Server 版本匹配。 + +当前 `dev` 示例按 commit [`f1a1a0a`](https://github.com/apache/seatunnel/commit/f1a1a0abbe24bdac8cf23307995a78f778a3f467) 核对，固定版本的 [HugeGraph Source 文档](https://github.com/apache/seatunnel/blob/f1a1a0abbe24bdac8cf23307995a78f778a3f467/docs/zh/connectors/source/HugeGraph.md) 和 [HugeGraph Sink 文档](https://github.com/apache/seatunnel/blob/f1a1a0abbe24bdac8cf23307995a78f778a3f467/docs/zh/connectors/sink/HugeGraph.md) 与本文对应。`dev` 会继续变化，使用新版本前请重新核对配置。 + +2.3.13 的 HugeGraph Sink 使用 `schema_config`，这个版本没有 HugeGraph Source。本文所有发布版示例都按这个边界编写。 + +## 2 选哪个工具 + +先看数据从哪里来，以及任务是否已经属于一条大数据管道。 + +| 你的任务 | 推荐工具 | 适合原因 | +| --- | --- | --- | +| 管理图、执行 Gremlin、备份恢复、图克隆 | [HugeGraph-Tools](/cn/docs/quickstart/toolchain/hugegraph-tools) | 只操作 HugeGraph，命令直接 | +| 把本地文件、HDFS、MySQL 等数据批量导入 HugeGraph | [HugeGraph-Loader](/cn/docs/quickstart/toolchain/hugegraph-loader) | 配置简单，导入流程短 | +| 数据要经过 Kafka、JDBC、Flink、Spark 或多个外部系统 | SeaTunnel | 可以复用已有数据管道 | +| 需要流式任务、checkpoint 或统一管理多个连接器 | SeaTunnel | 支持 Source、Transform 和 Sink 组合 | +| 稳定地把一张 HugeGraph 图复制到另一张图 | Tools 优先 | 发布版工具更直接；SeaTunnel Source 仍是 dev 预览 | + +只维护一张图、没有现成大数据管道时，优先从 Loader 或 Tools 开始。SeaTunnel 需要额外准备连接器插件，并使用 HOCON 配置文件。 + +## 3 准备工作 + +### 3.1 HugeGraph + +本文示例使用以下图模型。 + +| 图元素 | 配置 | +| --- | --- | +| VertexLabel | `person`，主键为 `name` | +| PropertyKey | `name` 为 Text，`age` 为 Int | +| EdgeLabel | `knows`，源和目标都是 `person`，属性为 `since` | + +2.3.13 的 Sink 会按 `schema_config` 读取已有的 VertexLabel、EdgeLabel 和 PropertyKey。运行写入任务前，请先在 Hubble、REST API 或 Gremlin 中创建 Schema。 + +### 3.2 SeaTunnel + +请按 [SeaTunnel 本地部署文档](https://seatunnel.apache.org/docs/getting-started/locally/deployment/) 获取发行包。2.2.0-beta 之后，发行包默认不带连接器依赖，需要按任务安装 JDBC、Kafka 和 HugeGraph 插件；JDBC 还需要对应数据库的驱动。 + +如果 SeaTunnel 与 HugeGraph 不在同一台机器，`host` 要填写 SeaTunnel 运行环境可以访问的地址。容器内的 `127.0.0.1` 指向 SeaTunnel 容器自身；同一 Docker 网络中的服务则使用 HugeGraph 的服务名。 + +## 4 sql2graph + +JDBC 方式适合把关系库中的表或 SQL 查询结果导入 HugeGraph。下面的例子把 `person` 表写成顶点，使用 `name` 生成 HugeGraph 主键。 + + + +### 4.1 关系库到顶点 + +假设 MySQL 中有一张表。 + +```sql +CREATE TABLE person ( + name VARCHAR(64) PRIMARY KEY, + age INT NOT NULL +); +``` + +在 SeaTunnel 安装目录下创建 `config/sql2graph-person.conf`。 + +```hocon +env { + job.mode = "BATCH" +} + +source { + Jdbc { + url = "jdbc:mysql://mysql:3306/demo?useSSL=false&serverTimezone=UTC" + driver = "com.mysql.cj.jdbc.Driver" + username = "seatunnel" + password = "change_me" + query = "SELECT name, age FROM person ORDER BY name" + } +} + +sink { + HugeGraph { + host = "hugegraph" + port = 8080 + graph_name = "hugegraph" + graph_space = "default" + schema_config = { + type = "VERTEX" + label = "person" + idStrategy = "PRIMARY_KEY" + idFields = ["name"] + properties = ["name", "age"] + } + } +} +``` + +执行任务。 + +```bash +./bin/seatunnel.sh --config ./config/sql2graph-person.conf -m local +``` + +执行后可以在 HugeGraph 中检查顶点。 + +```groovy +g.V().hasLabel('person').valueMap('name', 'age') +``` + +`Jdbc` 的 `url` 和 `driver` 必填。`username` 和 `password` 按数据库认证配置填写，匿名连接时可以省略；示例中的密码需要替换。MySQL 驱动需要放到 SeaTunnel 对应引擎的插件目录，具体位置见 [JDBC Source 文档](https://seatunnel.apache.org/docs/connectors/source/Jdbc/)。 + +### 4.2 关系库到边 + +如果关系表中的端点字段已经能直接对应 `person.name`，可以再运行一个边任务。假设表结构如下。 + +```sql +CREATE TABLE knows ( + source_name VARCHAR(64) NOT NULL, + target_name VARCHAR(64) NOT NULL, + since INT NOT NULL +); +``` + +<details> +<summary>展开查看边任务配置</summary> + +```hocon +env { + job.mode = "BATCH" +} + +source { + Jdbc { + url = "jdbc:mysql://mysql:3306/demo?useSSL=false&serverTimezone=UTC" + driver = "com.mysql.cj.jdbc.Driver" + username = "seatunnel" + password = "change_me" + query = "SELECT source_name, target_name, since FROM knows ORDER BY source_name, target_name" + } +} + +sink { + HugeGraph { + host = "hugegraph" + port = 8080 + graph_name = "hugegraph" + graph_space = "default" + schema_config = { + type = "EDGE" + label = "knows" + sourceConfig = { + label = "person" + idFields = ["source_name"] + } + targetConfig = { + label = "person" + idFields = ["target_name"] + } + properties = ["since"] + mapping = { + fieldMapping = { + source_name = "name" Review Comment: ⚠️ important - Separately from the missing `since` PropertyKey noted above, this job fails startup validation against the section 3.1 schema before any row is written. At 2.3.13, `HugeGraphSinkFactory.createSink` returns a `TableSink` that constructs `new HugeGraphSink(sinkConfig, context.getCatalogTable())`, and the `HugeGraphSink` constructor runs `new SchemaValidator(config, catalogTable.getSeaTunnelRowType()).validateSchema()`. `SchemaValidator.validateLabelProperties` loops over every field of the catalog row type, computes `propertyName = fieldMapping.getOrDefault(fieldName, fieldName)`, and throws `INVALID_GRAPH_SCHEMA` when that name is absent from `edgeLabel.properties()`: ``` "Property '%s' for label '%s' is defined in the connector config, but does not exist in the HugeGraph schema." ``` The JDBC query supplies `source_name`, `target_name` and `since`, and this `fieldMapping` sends both endpoint columns to `name`, which section 3.1 does not give `knows`. Note that the validator reads the full row type, so any edge job whose source carries endpoint ID columns hits this regardless of `fieldMapping`. Please declare `name` as a property of EdgeLabel `knows` in section 3.1 and say that the endpoint columns require it. `EdgeMapper` skips fields listed in `sourceConfig.idFields` / `targetConfig.idFields` when writing properties, so no stray `name` value lands on the edge; this is a workaround for a validator that inspects columns it does not write, not a graph modelling choice. ########## content/cn/docs/quickstart/toolchain/hugegraph-seatunnel-connector.md: ########## @@ -0,0 +1,410 @@ +--- +title: "HugeGraph-SeaTunnel Connector Quick Start" +linkTitle: "使用 SeaTunnel Connector 同步数据" +weight: 5 +--- + +SeaTunnel 负责连接数据源和数据目的地。HugeGraph Connector 提供两种能力。 + +- `HugeGraph Sink` 把文件、数据库、Kafka 等数据写入 HugeGraph。 +- `HugeGraph Source` 从 HugeGraph 读出顶点和边，已合入 `dev`（将随 SeaTunnel 3.0.0 发布），尚未随正式版本发布。 + + + +## 1 先看版本 + +本文把发布版和开发版分开写。配置放错版本，任务会在启动阶段失败。 + +| 使用内容 | SeaTunnel 版本 | 配置方式 | 状态 | +| --- | --- | --- | --- | +| JDBC / Kafka 写入 HugeGraph | 2.3.13 | `schema_config` | 发布版 | +| HugeGraph 读取和图迁移 | 当前 `dev` | HugeGraph Source + `mappings` | 已合入 dev，随 3.0.0 发布 | +| `graph2graph` | 当前 `dev` | Source + Sink | 已合入 dev，随 3.0.0 发布 | + +HugeGraph Source 与 `mappings` 多映射重构已随 PR [apache/seatunnel#11413](https://github.com/apache/seatunnel/pull/11413) 合入 dev（2026-08），将随 SeaTunnel **3.0.0** 正式发布。2.3.13 发行包内置 HugeGraph Client 1.5.0，dev 已升级到 1.7.0，需与 Server 版本匹配。 + +当前 `dev` 示例按 commit [`f1a1a0a`](https://github.com/apache/seatunnel/commit/f1a1a0abbe24bdac8cf23307995a78f778a3f467) 核对，固定版本的 [HugeGraph Source 文档](https://github.com/apache/seatunnel/blob/f1a1a0abbe24bdac8cf23307995a78f778a3f467/docs/zh/connectors/source/HugeGraph.md) 和 [HugeGraph Sink 文档](https://github.com/apache/seatunnel/blob/f1a1a0abbe24bdac8cf23307995a78f778a3f467/docs/zh/connectors/sink/HugeGraph.md) 与本文对应。`dev` 会继续变化，使用新版本前请重新核对配置。 + +2.3.13 的 HugeGraph Sink 使用 `schema_config`，这个版本没有 HugeGraph Source。本文所有发布版示例都按这个边界编写。 + +## 2 选哪个工具 + +先看数据从哪里来，以及任务是否已经属于一条大数据管道。 + +| 你的任务 | 推荐工具 | 适合原因 | +| --- | --- | --- | +| 管理图、执行 Gremlin、备份恢复、图克隆 | [HugeGraph-Tools](/cn/docs/quickstart/toolchain/hugegraph-tools) | 只操作 HugeGraph，命令直接 | +| 把本地文件、HDFS、MySQL 等数据批量导入 HugeGraph | [HugeGraph-Loader](/cn/docs/quickstart/toolchain/hugegraph-loader) | 配置简单，导入流程短 | +| 数据要经过 Kafka、JDBC、Flink、Spark 或多个外部系统 | SeaTunnel | 可以复用已有数据管道 | +| 需要流式任务、checkpoint 或统一管理多个连接器 | SeaTunnel | 支持 Source、Transform 和 Sink 组合 | +| 稳定地把一张 HugeGraph 图复制到另一张图 | Tools 优先 | 发布版工具更直接；SeaTunnel Source 仍是 dev 预览 | + +只维护一张图、没有现成大数据管道时，优先从 Loader 或 Tools 开始。SeaTunnel 需要额外准备连接器插件，并使用 HOCON 配置文件。 + +## 3 准备工作 + +### 3.1 HugeGraph + +本文示例使用以下图模型。 + +| 图元素 | 配置 | +| --- | --- | +| VertexLabel | `person`，主键为 `name` | +| PropertyKey | `name` 为 Text，`age` 为 Int | +| EdgeLabel | `knows`，源和目标都是 `person`，属性为 `since` | + +2.3.13 的 Sink 会按 `schema_config` 读取已有的 VertexLabel、EdgeLabel 和 PropertyKey。运行写入任务前，请先在 Hubble、REST API 或 Gremlin 中创建 Schema。 + +### 3.2 SeaTunnel + +请按 [SeaTunnel 本地部署文档](https://seatunnel.apache.org/docs/getting-started/locally/deployment/) 获取发行包。2.2.0-beta 之后，发行包默认不带连接器依赖，需要按任务安装 JDBC、Kafka 和 HugeGraph 插件；JDBC 还需要对应数据库的驱动。 + +如果 SeaTunnel 与 HugeGraph 不在同一台机器，`host` 要填写 SeaTunnel 运行环境可以访问的地址。容器内的 `127.0.0.1` 指向 SeaTunnel 容器自身；同一 Docker 网络中的服务则使用 HugeGraph 的服务名。 + +## 4 sql2graph + +JDBC 方式适合把关系库中的表或 SQL 查询结果导入 HugeGraph。下面的例子把 `person` 表写成顶点，使用 `name` 生成 HugeGraph 主键。 + + + +### 4.1 关系库到顶点 + +假设 MySQL 中有一张表。 + +```sql +CREATE TABLE person ( + name VARCHAR(64) PRIMARY KEY, + age INT NOT NULL +); +``` + +在 SeaTunnel 安装目录下创建 `config/sql2graph-person.conf`。 + +```hocon +env { + job.mode = "BATCH" +} + +source { + Jdbc { + url = "jdbc:mysql://mysql:3306/demo?useSSL=false&serverTimezone=UTC" + driver = "com.mysql.cj.jdbc.Driver" + username = "seatunnel" + password = "change_me" + query = "SELECT name, age FROM person ORDER BY name" + } +} + +sink { + HugeGraph { + host = "hugegraph" + port = 8080 + graph_name = "hugegraph" + graph_space = "default" + schema_config = { + type = "VERTEX" + label = "person" + idStrategy = "PRIMARY_KEY" + idFields = ["name"] + properties = ["name", "age"] Review Comment: 🧹 minor - `properties` inside `schema_config` has no effect at 2.3.13, so this line reads as a write whitelist while writing nothing of the sort. Across all 18 main-source files of `connector-hugegraph` at tag 2.3.13, `SchemaConfig.getProperties` is never called: the only `properties()` calls are `vertexLabel.properties()` and `edgeLabel.properties()` in `SchemaValidator`, which read the server-side schema. `VertexMapper` and `EdgeMapper` write every field of the row type except the id fields and nulls, narrowed only by `selected_fields` / `ignored_fields`. Upstream's 2.3.13 example shows the key too, so keeping it is defensible, but please add a line to section 7 saying that `properties` is inert in 2.3.13 and that `selected_fields` / `ignored_fields` are what limit the written columns. The same applies at lines 176 and 241. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
