This is an automated email from the ASF dual-hosted git repository.
davidzollo pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/seatunnel-website.git
The following commit(s) were added to refs/heads/main by this push:
new 585c871e775d [Blog] Add bilingual EtLT data pipeline article (#488)
585c871e775d is described below
commit 585c871e775dc71c00a128a3d63366d4aacc7356
Author: Daniel <[email protected]>
AuthorDate: Sun Jul 19 21:00:52 2026 +0800
[Blog] Add bilingual EtLT data pipeline article (#488)
---
.../2026-07-19-etlt-middle-ground-data-pipeline.md | 216 +++++++++++++++++++++
.../2026-07-19-etlt-middle-ground-data-pipeline.md | 216 +++++++++++++++++++++
2 files changed, 432 insertions(+)
diff --git a/blog/2026-07-19-etlt-middle-ground-data-pipeline.md
b/blog/2026-07-19-etlt-middle-ground-data-pipeline.md
new file mode 100644
index 000000000000..c8f5b326efe9
--- /dev/null
+++ b/blog/2026-07-19-etlt-middle-ground-data-pipeline.md
@@ -0,0 +1,216 @@
+---
+slug: etlt-middle-ground-data-pipeline
+title: "EtLT: The Middle Ground Your Data Pipeline Has Been Missing"
+tags: [SeaTunnel, Architecture, "Data Integration", EtLT, CDC, dbt]
+authors:
+ - name: David Zollo
+ title: Apache SeaTunnel Contributor
+ url: https://github.com/davidzollo
+ image_url: https://github.com/davidzollo.png
+---
+
+# EtLT: The Middle Ground Your Data Pipeline Has Been Missing
+
+Ask a data engineer whether their pipeline is ETL or ELT and you'll get an
instant answer. Old-school engineers say ETL. dbt users say ELT.
+
+Both answers are incomplete. There's a third pattern that more accurately
describes what modern data pipelines actually do — and it's been hiding in
plain sight: **EtLT**.
+
+---
+
+## The Three Paradigms
+
+### ETL: Transform Before You Land
+
+Raw data is extracted, passed through a dedicated transformation tier (Spark,
DataStage, Informatica), and only then written to the destination warehouse.
+
+```
+Source ──► [Transform tier] ──► Destination
+```
+
+**Pros:** The warehouse always holds clean, business-ready data. Compliance
controls are enforced before data ever lands.
+**Cons:** The transform tier becomes a bottleneck. Schema changes require
coordinated updates across multiple layers. Running a dedicated compute cluster
for transforms is expensive.
+
+### ELT: Land First, Transform In-Place
+
+Popularized by dbt. Raw data lands directly in the warehouse (BigQuery,
Snowflake, ClickHouse), and SQL does the transformations in place. The
transform "tier" is just the warehouse itself.
+
+```
+Source ──► Destination (raw layer) ──► [SQL inside warehouse] ──► Business
layer
+```
+
+**Pros:** Raw data is preserved for auditing. You reuse warehouse compute.
Iteration is fast.
+**Cons:** Sensitive fields — SSNs, email addresses, phone numbers — land in
plaintext. There's no second chance to mask them once they're in the warehouse.
Data quality issues only surface after Load, at which point downstream models
may already be contaminated.
+
+### EtLT: A Lightweight Transform In-Transit
+
+```
+Source ──► [tiny t] ──► Destination (sanitized raw layer) ──► [T inside
warehouse]
+```
+
+The **tiny t** is a small set of row-level transformations that happen while
data is in flight:
+
+| tiny t operation | Purpose |
+|---|---|
+| Field projection / column pruning | Drop unused columns before transfer —
save bandwidth |
+| PII masking | Phone numbers, SSNs, emails are anonymized before landing —
compliance enforced at the pipeline layer, not as an afterthought |
+| Type normalization | Source `VARCHAR "2023-01-01"` becomes `DATE` on arrival
— no type-casting SQL needed in the warehouse |
+| Row filtering | Unwanted events can be discarded pre-Load; stateful CDC
transitions require changelog-aware handling |
+| Field renaming | Align to destination naming conventions without an alias
layer |
+| NULL backfill | Reduce `COALESCE` calls in downstream aggregation SQL |
+
+The **big T** (post-Load Transform) is where actual business logic lives:
multi-table JOINs, metric calculations, aggregations, ML feature engineering.
+
+---
+
+## Why Engineers Keep Overlooking EtLT
+
+The reason is tooling — not concept.
+
+Legacy ETL tools (Informatica, DataStage) made transformations expensive and
complex. Engineers overcorrected by pushing all logic into the pipeline, which
made pipelines brittle.
+
+Modern ELT tools (Airbyte, Fivetran) swung to the opposite extreme: move data
from source to destination with almost no in-transit processing, then let dbt
handle everything.
+
+**The gap neither camp fills:** when you need both in-transit operations
(masking, filtering) *and* complex analytical SQL in the destination, you end
up with awkward workarounds in both tool families.
+
+EtLT fills exactly that gap.
+
+---
+
+## SeaTunnel Is Built for EtLT
+
+Apache SeaTunnel describes itself in its official documentation as an **"EL(T)
data integration platform"** — the parentheses around T are intentional. The
Transform step is lightweight and optional. This is not a marketing choice;
it's an architectural constraint.
+
+### The Three-Layer Model Maps Directly to EtLT
+
+SeaTunnel's execution model has three stages: **Source → Transform → Sink**.
+
+```
+Source (E)
+ └──► Transform (tiny t) ← optional, lightweight processing
+ └──► Sink (L)
+ └──► [Warehouse SQL / dbt] (T) ← big T lives here
+```
+
+SeaTunnel draws a clear line around what Transform can do. The official docs
state:
+
+> "Transform can only be used for some simple transformations of data, such as
converting a column to uppercase/lowercase, modifying column names, or
splitting one column into multiple columns."
+
+This describes the intended scope of the built-in transforms. In the current
Zeta SQL Transform, JOIN and GROUP BY are not supported, so cross-table joins
and aggregations belong in the destination system or another dedicated
processing layer.
+
+### Built-in Transforms Cover the Typical tiny t Operations
+
+| SeaTunnel Transform | tiny t operation |
+|---|---|
+| `FieldMapper` | Column renaming, field projection |
+| `Filter` | Column projection with include/exclude lists |
+| `Replace` | Field value substitution (masking/redaction) |
+| `Split` | Split one column into multiple (e.g., address parsing) |
+| `SQL Transform` | Row filtering and lightweight SQL expressions; current
Zeta implementation does not support JOIN or GROUP BY |
+| `Copy` | Field duplication |
+
+SeaTunnel provides both map and flat-map transform interfaces. Its current
built-in transforms focus on processing individual records and schemas rather
than cross-row aggregation, which makes them a practical fit for the tiny t
layer.
+
+### Where EtLT Matters Most: CDC Pipelines
+
+Real-time CDC sync is one of SeaTunnel's core use cases — and it's also where
pure ELT breaks down for compliance.
+
+Here's the fundamental problem: **once a value from a binlog event lands in
your warehouse, there is no second chance to mask it.** A phone number or SSN
written to a ClickHouse table can't be un-written by a downstream dbt model.
The original plaintext is already persisted.
+
+EtLT solves what ELT cannot: it applies compliance transformations inside the
only window that exists before the data reaches its destination.
+
+A minimal configuration shape for a SeaTunnel CDC job with tiny t transforms
is shown below. Replace the example credentials and endpoints before running it:
+
+```hocon
+env {
+ job.mode = "STREAMING"
+}
+
+source {
+ MySQL-CDC {
+ plugin_output = "raw_user_info"
+ url = "jdbc:mysql://localhost:3306/orders"
+ username = "seatunnel"
+ password = "change-me"
+ server-id = 5601-5604
+ table-names = ["orders.user_info"]
+ }
+}
+
+transform {
+ # Mask phone numbers in-transit — the raw value never reaches the warehouse
+ Replace {
+ plugin_input = "raw_user_info"
+ plugin_output = "masked_user_info"
+ replace_field = "phone"
+ pattern = "(\\d{3})\\d{4}(\\d{4})"
+ replacement = "$1****$2"
+ is_regex = true
+ }
+}
+
+sink {
+ # L: land into the ClickHouse raw layer
+ Clickhouse {
+ plugin_input = "masked_user_info"
+ host = "clickhouse-host:8123"
+ database = "raw"
+ table = "user_info"
+ username = "default"
+ password = "change-me"
+ primary_key = "id"
+ support_upsert = true
+ allow_experimental_lightweight_delete = true
+ }
+}
+```
+
+Once the data is in ClickHouse, you build wide tables, compute retention
metrics, and run analytical queries — that's the big T. dbt models are a
natural fit here.
+
+---
+
+## SeaTunnel + dbt: Full-Stack EtLT
+
+dbt owns the big T after Load. SeaTunnel owns everything from source to Load
(including tiny t). They're complementary, not competing.
+
+```
+Source
+ └── SeaTunnel (E + tiny t + L)
+ └── dbt (T)
+ └── BI / ML
+```
+
+SeaTunnel handles data arrival and in-transit hygiene. dbt handles data
modeling and business logic. Each tool does what it's good at and nothing more.
+
+---
+
+## The Trap: Trying to Push Big T Into SeaTunnel
+
+Because SeaTunnel supports `SQL Transform`, engineers sometimes try writing
`GROUP BY` aggregations there. The current Zeta SQL Transform explicitly
rejects GROUP BY and JOIN queries.
+
+If you need pre-Load aggregation, you have two real options:
+
+1. **Do it in the destination system** — this is the whole point of EtLT and
ELT.
+2. **Use a dedicated processing job outside the SeaTunnel transform chain** —
at that point you are building a full streaming ETL pipeline rather than
keeping the transformation in the tiny t layer.
+
+Clear boundaries make failures easier to trace. When tiny t and big T are
collapsed into the same layer, it becomes nearly impossible to reason about
where something went wrong.
+
+---
+
+## Summary
+
+| Pattern | Best fit | SeaTunnel's role |
+|---|---|---|
+| ETL | Strong compliance requirements, dedicated transform compute cluster |
Handles E and L plus supported lightweight transforms; complex T requires a
dedicated processing layer |
+| ELT | Destination is a powerful SQL engine (BigQuery/Snowflake), no strict
in-transit compliance requirements | Pure E + L — disable Transform |
+| **EtLT** | **Real-time CDC, in-transit PII masking, destination has dbt/SQL
capability** | **Natural fit: E + tiny t + L; big T belongs in the
destination** |
+
+SeaTunnel is a natural fit for EtLT — not because it claims the label, but
because its Source/Transform/Sink separation, current built-in transform scope,
and explicit "EL(T)" positioning all point to the same shape: move data fast,
sanitize lightly in-flight, and leave the heavy lifting to the destination.
+
+---
+
+## Further Reading
+
+- [Apache SeaTunnel —
About](https://seatunnel.apache.org/docs/2.3.13/introduction/about/#seatunnel-work-flowchart)
— official EL(T) positioning (see the "SeaTunnel Work Flowchart" section)
+- [SeaTunnel Transform
Connectors](https://seatunnel.apache.org/docs/transforms/) — full list of
built-in lightweight transforms
+- [dbt](https://www.getdbt.com/) — the natural partner for the big T in EtLT
+- [SeaTunnel API Deep
Dive](https://seatunnel.apache.org/blog/2026/01/06/seatunnel-api-deep-dive) —
deep dive into the single-row `map` contract
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-blog/2026-07-19-etlt-middle-ground-data-pipeline.md
b/i18n/zh-CN/docusaurus-plugin-content-blog/2026-07-19-etlt-middle-ground-data-pipeline.md
new file mode 100644
index 000000000000..53ca4b61fef2
--- /dev/null
+++
b/i18n/zh-CN/docusaurus-plugin-content-blog/2026-07-19-etlt-middle-ground-data-pipeline.md
@@ -0,0 +1,216 @@
+---
+slug: etlt-middle-ground-data-pipeline
+title: "EtLT：你的数据管道一直在用的模式，只是没有名字"
+tags: [SeaTunnel, 架构, 数据集成, EtLT, CDC, dbt]
+authors:
+ - name: David Zollo
+ title: Apache SeaTunnel 贡献者
+ url: https://github.com/davidzollo
+ image_url: https://github.com/davidzollo.png
+---
+
+# EtLT：你的数据管道一直在用的模式，只是没有名字
+
+问一位数据工程师他的管道是 ETL 还是 ELT，几乎总能得到秒答——老派工程师说 ETL，dbt 用户说 ELT。
+
+但这两个答案都不完整。有第三种模式更准确地描述了现代数据管道实际在做的事——它一直存在，只是没有被命名：**EtLT**。
+
+---
+
+## 三种范式
+
+### ETL：落库前先转换
+
+原始数据从源端提取后，经过专用转换层（Spark、DataStage、Informatica）处理，再写入目标数仓。
+
+```
+源端 ──► [转换层] ──► 目标端
+```
+
+**优点：** 目标端始终存放干净、业务就绪的数据。合规管控在数据落库前就已强制执行。
+**缺点：** 转换层容易成为瓶颈。Schema 变更需要跨多层同步修改。专用计算集群的运维成本高。
+
+### ELT：先落库，原地转换
+
+由 dbt 带火的模式。原始数据直接落入数仓（BigQuery、Snowflake、ClickHouse），再用 SQL
原地做转换。转换层不再是独立集群——就是数仓本身。
+
+```
+源端 ──► 目标端（原始层）──► [数仓内 SQL] ──► 业务层
+```
+
+**优点：** 原始数据保留完整审计价值。复用数仓算力。迭代速度快。
+**缺点：** 手机号、身份证、邮箱等敏感字段以明文落库，存在合规风险。数据质量问题要等 Load 之后才可见，此时下游模型可能已经被污染。
+
+### EtLT：传输途中做轻量变换
+
+```
+源端 ──► [tiny t] ──► 目标端（净化原始层）──► [目标端内的 T]
+```
+
+**tiny t** 是数据在传输过程中完成的一组行级变换：
+
+| tiny t 操作 | 目的 |
+|---|---|
+| 字段投影 / 列裁剪 | 传输前删除无用列——节省带宽 |
+| PII 脱敏 | 手机号、身份证、邮箱在落库前匿名化——合规在管道层强制落地，而非事后补救 |
+| 类型归一 | 源端 `VARCHAR "2023-01-01"` 在入仓时就转为 `DATE`——省去数仓侧的类型转换 SQL |
+| 行过滤 | 不需要的事件可在 Load 前丢弃；有状态的 CDC 变化需要按 changelog 语义处理 |
+| 字段重命名 | 适配目标端命名规范，无需数仓侧的别名层 |
+| NULL 回填 | 减少下游聚合 SQL 中的 `COALESCE` 调用 |
+
+**大 T**（Load 后的 Transform）才是真正的业务逻辑所在：多表 JOIN、指标计算、聚合、ML 特征工程。
+
+---
+
+## 为什么工程师长期忽略 EtLT
+
+根本原因是工具，不是概念本身。
+
+传统 ETL 工具（Informatica、DataStage）把转换做得极重，工程师被迫把大量业务逻辑塞进管道，结果管道变得脆弱、难以维护。
+
+现代 ELT 工具（Airbyte、Fivetran）走向另一个极端：源端到目标端几乎没有在途处理，全部交给 dbt。
+
+**两类工具都没能填好的空白：** 当你同时需要在途操作（脱敏、过滤）*和*目标端的复杂分析 SQL 时，两个工具族都会让你陷入尴尬的临时方案。
+
+EtLT 天然填补这个空白。
+
+---
+
+## SeaTunnel 天然契合 EtLT
+
+Apache SeaTunnel 在官方文档中把自己定位为 **"EL(T) 数据集成工具"**——括号里的 T 是刻意为之。Transform
步骤是轻量的、可选的。这不是营销话术，而是一个架构约束。
+
+### 三层模型与 EtLT 直接对应
+
+SeaTunnel 的执行模型分三层：**Source → Transform → Sink**。
+
+```
+Source（E）
+ └──► Transform（tiny t） ← 可选，轻量处理
+ └──► Sink（L）
+ └──► [数仓 SQL / dbt]（T） ← 大 T 在这里
+```
+
+SeaTunnel 对 Transform 能做的事划定了清晰边界。官方文档写道：
+
+> "Transform can only be used for some simple transformations of data, such as
converting a column to uppercase/lowercase, modifying column names, or
splitting one column into multiple columns."
+
+这描述的是内置 Transform 的目标边界。当前 Zeta SQL Transform 不支持 JOIN 和 GROUP BY，因此跨表 JOIN
与聚合应放在目标系统或其他专用处理层中完成。
+
+### 内置 Transform 覆盖典型 tiny t 需求
+
+| SeaTunnel Transform | 对应的 tiny t 操作 |
+|---|---|
+| `FieldMapper` | 字段重命名、列投影 |
+| `Filter` | 通过 include/exclude 列表做字段投影 |
+| `Replace` | 字段值替换（脱敏 / 打码） |
+| `Split` | 将一列拆成多列（如地址解析） |
+| `SQL Transform` | 行过滤和轻量 SQL 表达式；当前 Zeta 实现不支持 JOIN 或 GROUP BY |
+| `Copy` | 字段复制 |
+
+SeaTunnel 同时提供 map 和 flat-map Transform 接口。当前内置 Transform 主要处理单条记录和
Schema，而不是跨行聚合，因此很适合作为 tiny t 层。
+
+### EtLT 最关键的战场：CDC 管道
+
+实时 CDC 同步是 SeaTunnel 的核心使用场景之一——也是纯 ELT 在合规上失守的地方。
+
+根本问题在于：**binlog 事件里的字段值一旦落库，就没有第二次机会打码了。** 写入 ClickHouse 的手机号或身份证，不可能被下游 dbt
模型"反写"回来。原始明文已经持久化。
+
+EtLT 解决了 ELT 无法解决的问题：在数据到达目标端之前的唯一时间窗口内，完成合规变换。
+
+下面给出一个带 tiny t 的 SeaTunnel CDC 作业最小配置形态。运行前请替换示例中的账号、密码和服务地址：
+
+```hocon
+env {
+ job.mode = "STREAMING"
+}
+
+source {
+ MySQL-CDC {
+ plugin_output = "raw_user_info"
+ url = "jdbc:mysql://localhost:3306/orders"
+ username = "seatunnel"
+ password = "change-me"
+ server-id = 5601-5604
+ table-names = ["orders.user_info"]
+ }
+}
+
+transform {
+ # 在传输途中打码手机号——原始值永远不会到达数仓
+ Replace {
+ plugin_input = "raw_user_info"
+ plugin_output = "masked_user_info"
+ replace_field = "phone"
+ pattern = "(\\d{3})\\d{4}(\\d{4})"
+ replacement = "$1****$2"
+ is_regex = true
+ }
+}
+
+sink {
+ # L：落入 ClickHouse 原始层
+ Clickhouse {
+ plugin_input = "masked_user_info"
+ host = "clickhouse-host:8123"
+ database = "raw"
+ table = "user_info"
+ username = "default"
+ password = "change-me"
+ primary_key = "id"
+ support_upsert = true
+ allow_experimental_lightweight_delete = true
+ }
+}
+```
+
+数据进入 ClickHouse 后，再用 SQL 构建宽表、计算留存指标、做分析查询——这就是大 T。dbt 模型在这里是天然的搭档。
+
+---
+
+## SeaTunnel + dbt：全栈 EtLT
+
+dbt 负责 Load 之后的大 T；SeaTunnel 负责从源到 Load（含 tiny t）。两者互补，而非竞争。
+
+```
+源端
+ └── SeaTunnel（E + tiny t + L）
+ └── dbt（T）
+ └── BI / ML
+```
+
+SeaTunnel 负责数据到达和在途卫生；dbt 负责数据建模和业务逻辑。每个工具只做它擅长的事。
+
+---
+
+## 常见陷阱：把大 T 塞进 SeaTunnel
+
+因为 SeaTunnel 支持 `SQL Transform`，有些工程师会尝试在里面写 `GROUP BY` 聚合。当前 Zeta SQL
Transform 会明确拒绝 GROUP BY 和 JOIN 查询。
+
+如果确实需要 Load 前的聚合，有两条路：
+
+1. **在目标系统里做**——这本来就是 EtLT 和 ELT 中大 T 的用途。
+2. **在 SeaTunnel Transform 链路之外使用专用处理作业**——此时构建的是完整流式 ETL 管道，而不是把转换留在 tiny t 层。
+
+边界清晰，失败才容易定位。把 tiny t 和大 T 混在同一层，出问题时几乎无法推断根源在哪里。
+
+---
+
+## 总结
+
+| 范式 | 最适合的场景 | SeaTunnel 的角色 |
+|---|---|---|
+| ETL | 强合规要求、有专用转换计算集群 | 负责 E、L 和已支持的轻量转换；复杂 T 需要专用处理层 |
+| ELT | 目标端是强 SQL 引擎（BigQuery/Snowflake），无严格在途合规要求 | 纯 E + L——关闭 Transform |
+| **EtLT** | **实时 CDC、在途 PII 脱敏、目标端有 dbt/SQL 能力** | **天然契合：E + tiny t + L；大 T
归目标端** |
+
+SeaTunnel 天然适合 EtLT——不是因为它声称这个标签，而是因为 Source/Transform/Sink 三层分离、当前内置
Transform 的能力边界和明确的"EL(T)"定位都指向同一件事：快速移动数据，在传输途中轻量净化，把重活留给目标系统。
+
+---
+
+## 延伸阅读
+
+- [Apache SeaTunnel —
About](https://seatunnel.apache.org/docs/2.3.13/introduction/about/#seatunnel-work-flowchart)
— 官方 EL(T) 定位说明（在页面 "SeaTunnel Work Flowchart" 小节）
+- [SeaTunnel Transform
Connectors](https://seatunnel.apache.org/docs/transforms/) — 内置轻量 Transform 算子一览
+- [dbt](https://www.getdbt.com/) — EtLT 中大 T 的标准搭档
+- [SeaTunnel API Deep
Dive](https://seatunnel.apache.org/blog/2026/01/06/seatunnel-api-deep-dive) —
Transform map 接口单行约束详解
