This is an automated email from the ASF dual-hosted git repository.
vinothchandar pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/asf-site by this push:
new 7d351ca89254 docs(blog): Delta/Iceberg migration guides and CDC
comparison (#19272)
7d351ca89254 is described below
commit 7d351ca8925424c671d78a3e83743d8f860faad0
Author: vinoth chandar <[email protected]>
AuthorDate: Thu Aug 6 19:50:27 2026 +0530
docs(blog): Delta/Iceberg migration guides and CDC comparison (#19272)
---
...2026-08-04-migrating-from-delta-lake-to-hudi.md | 148 +++++++++++++++++++++
...-08-05-migrating-from-apache-iceberg-to-hudi.md | 139 +++++++++++++++++++
...2026-08-06-hudi-vs-iceberg-for-cdc-workloads.md | 110 +++++++++++++++
website/docs/migration_guide.md | 7 +
website/static/llms.txt | 3 +
5 files changed, 407 insertions(+)
diff --git a/website/blog/2026-08-04-migrating-from-delta-lake-to-hudi.md
b/website/blog/2026-08-04-migrating-from-delta-lake-to-hudi.md
new file mode 100644
index 000000000000..31f5d4c64fb0
--- /dev/null
+++ b/website/blog/2026-08-04-migrating-from-delta-lake-to-hudi.md
@@ -0,0 +1,148 @@
+---
+title: "Migrating from Delta Lake to Apache Hudi"
+excerpt: "A practical guide to moving Delta Lake tables to Apache Hudi —
metadata-only conversion with Apache XTable, a one-time Spark rewrite,
validation, and rollback."
+description: "How to migrate Delta Lake tables to Apache Hudi: XTable metadata
translation with no data rewrite, or a one-time Spark bulk insert — plus
validation and rollback."
+authors: [sivabalan]
+category: how-to
+image:
/assets/images/blog/2023-08-09-Lakehouse-Trifecta-Delta-Lake-Apache-Iceberg-and-Apache-Hudi.png
+tags:
+- migration
+- apache xtable
+- delta lake
+- guide
+---
+
+You can migrate a Delta Lake table to Apache Hudi either by translating its
metadata with [Apache XTable](https://xtable.apache.org) — no data rewrite
required, since both formats store data as Apache Parquet — or by rewriting the
table once with a Spark `bulk_insert`; many teams start by running both formats
side by side via XTable and cut writers over only after validating the Hudi
side.
+
+That single sentence is the whole decision in miniature, but a production
migration deserves more care than a summary. This guide walks through why teams
make this move, the three migration strategies and when each fits, the exact
XTable and Spark commands involved, the Delta-specific features that need
per-table attention, and how to validate and — if necessary — roll back.
Throughout, the framing to keep in mind is that both Delta Lake and Hudi are
[open table formats](/blog/2026/07/14 [...]
+
+## Why Teams Move from Delta Lake to Hudi
+
+Delta Lake takes a deliberately simple approach on disk and is closely
integrated with Spark. Teams that migrate to Hudi are usually not fleeing Delta
so much as reaching for write-side machinery that Hudi builds in:
+
+- **Record-level indexing.** Hudi maintains a [multi-modal indexing
subsystem](/docs/indexes) — record-level indexes, bloom filters, column
statistics, expression indexes — inside an internal metadata table. For
update-heavy workloads such as CDC ingestion, an index that maps record keys to
file groups means the writer can locate the files affected by an update without
scanning or joining against the whole table.
+- **Streaming-first Merge-on-Read design.** Hudi's MOR table type absorbs
updates into compact log files that are compacted asynchronously, decoupling
write latency from file rewrite cost. Workloads that need minute-level
freshness under continuous upserts tend to be the strongest motivation for the
move.
+- **Built-in table services.** Compaction, clustering, cleaning, and indexing
ship with the project and run inline or asynchronously, without a separate
orchestration layer or a commercial service to keep tables healthy.
+- **Built-in ingestion tooling.** Hudi Streamer provides a self-contained
ingestion utility with sources for Kafka, DFS, and JDBC, checkpoint management,
transformations, and catalog syncing.
+
+These differences are measurable, not just architectural. In benchmarks run
with the open-source [LakeLoader](https://github.com/onehouseinc/lake-loader)
framework — Spark 3.5, Hudi 1.1.1, Delta Lake 3.3.2, on S3 — we observed
roughly 6× lower incremental write latency for Hudi on a 10 TB partitioned fact
table with skewed updates, and about 5× lower steady-state latency on
Merge-on-Read tables taking sparse, column-level updates. The workload
definitions are format-agnostic and repeatab [...]
+
+None of this is a knock on Delta — if your workload is mostly appends with
occasional merges, run entirely on Spark, and you are happy with your current
operational model, you may not need to migrate at all. The rest of this guide
assumes you have concluded the write-side capabilities matter for your workload.
+
+## Understand the Three Migration Options
+
+| | A. XTable metadata translation | B. Full rewrite (Spark bulk insert) | C.
Incremental dual-write cutover |
+|---|---|---|---|
+| **Data movement** | None — metadata only | Full copy of the table | Full
copy, spread over time |
+| **Downtime for writers** | None during sync; brief pause at cutover | Pause
writes during the rewrite (or reconcile a delta) | None — new writer runs in
parallel |
+| **Resulting table** | Hudi metadata over existing Parquet files | Native
Hudi table, freshly laid out | Native Hudi table |
+| **Lets you re-key / re-partition / resize files** | No — inherits Delta's
layout | Yes | Yes |
+| **Reversible** | Trivially — source metadata untouched | Source table left
intact until decommission | Source table left intact until decommission |
+| **Best for** | Large tables, fast side-by-side evaluation, low-risk cutover
| Small-to-medium tables, or when you want a clean re-layout | Very large, hot
tables that cannot pause and need a new layout |
+
+A few rules of thumb. If the table is large and its current Parquet layout is
acceptable, start with **Option A** — it costs almost nothing to try and keeps
both formats readable while you evaluate. If the table is small enough that a
rewrite finishes in an acceptable window, or you want to change record keys,
partitioning, or file sizes as part of the move, **Option B** is simpler to
reason about. **Option C** — standing up a parallel Hudi pipeline fed from the
same upstream source, bac [...]
+
+## Option A: Convert with Apache XTable
+
+[Apache XTable](https://xtable.apache.org) (incubating) is an open source
project that translates table metadata between Delta Lake, Hudi, and Iceberg in
any direction, without copying or rewriting data files. For this migration,
Delta is the source and Hudi is the target. XTable reads the Delta transaction
log and writes out the equivalent Hudi metadata — schema, commit history,
partition information, and column statistics — alongside the existing Parquet
files. (The same tool also work [...]
+
+Create a config file describing the source and target:
+
+```yaml
+# my_config.yaml
+sourceFormat: DELTA
+targetFormats:
+ - HUDI
+datasets:
+ - tableBasePath: s3://bucket/warehouse/orders
+ tableName: orders
+```
+
+Then run the sync with the bundled XTable jar (built from
[source](https://github.com/apache/incubator-xtable) or downloaded from the
project's GitHub packages):
+
+```shell
+java -jar path/to/xtable-utilities-bundled.jar --datasetConfig my_config.yaml
+```
+
+When the sync completes, the table's base path contains a `.hoodie` directory
with Hudi's timeline and metadata, side by side with Delta's `_delta_log`. No
Parquet file was read or written — the job's runtime scales with the amount of
metadata (number of files and commits), not with data volume. The same
directory is now readable as a Delta table *and* as a Hudi table.
+
+Two operational notes, faithful to the [XTable
documentation](https://xtable.apache.org/docs/how-to):
+
+- **Syncs are repeatable and incremental.** XTable supports incremental sync
(translating only new commits since the last run) with a fallback to full sync,
so you can run it on a schedule — or after each Delta commit — to keep the Hudi
metadata current while the Delta writer keeps running.
+- **Catalog registration is a separate step.** XTable produces metadata in
storage; to query the table as Hudi from your engines, register it in your
catalog (Hive Metastore, AWS Glue) using Hudi's catalog sync tools or XTable's
own catalog sync support. Hudi's [XTable page](/docs/syncing_xtable) shows the
reverse direction and the Hudi Streamer integration.
+
+## The Catch: Converted vs Native Tables
+
+Here is the honest fine print. A converted table is *readable* as a Hudi table
— snapshot queries, engine integrations, and catalog syncing all work. But most
of the reasons you are migrating live on the **write path**: record-level
indexes are built and maintained by Hudi writers; streaming upserts, MOR log
files, and table services all require Hudi to be the one committing to the
table. XTable gives you a Hudi-readable table; it does not retroactively give
your Delta writer Hudi's writ [...]
+
+So a metadata conversion is the first half of the migration, not the whole
thing. The second half is the writer cutover, which follows a simple sequence:
+
+1. **Stop the Delta writer.** Pause the job or pipeline committing to the
Delta table. In-flight data can queue upstream (e.g., in Kafka) during the
brief window.
+2. **Run a final XTable sync.** Translate the last Delta commits so the Hudi
metadata reflects the table's final Delta-written state.
+3. **Start the Hudi writer.** Point your pipeline — Spark structured
streaming, Hudi Streamer, or batch jobs following the [quick start
guide](/docs/quick-start-guide) — at the same base path, configured with the
record key, ordering field, and table type you validated beforehand. From this
commit forward, Hudi owns the table and begins building its indexes and running
table services on new data.
+
+Until step 1, you can run the two formats side by side indefinitely: Delta
writers keep writing, XTable keeps both metadata layers in sync, and your
Hudi-native engines and pipelines read the converted table. That side-by-side
period is where the de-risking happens — you validate reads, permissions,
catalog integration, and downstream jobs against real data before any writer
changes.
+
+## Option B: Full Rewrite with Spark
+
+If the table is modest in size, or you want to change its physical layout —
different partitioning, tuned file sizes, a proper record key for upserts — a
one-time rewrite is the simplest path. Read the Delta table with Spark, write
it back as Hudi using `bulk_insert`, the write operation designed for exactly
this initial-load case:
+
+```scala
+// spark-shell with both Delta and Hudi bundles on the classpath
+val df = spark.read.format("delta").load("s3://bucket/warehouse/orders")
+
+df.write.format("hudi").
+ option("hoodie.datasource.write.recordkey.field", "order_id").
+ option("hoodie.datasource.write.partitionpath.field", "order_date").
+ option("hoodie.table.ordering.fields", "updated_at").
+ option("hoodie.datasource.write.operation", "bulk_insert").
+ option("hoodie.table.name", "orders").
+ mode("overwrite").
+ save("s3://bucket/warehouse/orders_hudi")
+```
+
+Note that the rewrite lands in a *new* base path, leaving the Delta table
untouched — that is your rollback story. Choose the record key and ordering
field deliberately here: the record key drives Hudi's indexing and upsert
semantics, and the ordering field resolves conflicts between multiple versions
of the same record (essential for CDC-style sources). Both are covered in the
[quick start guide](/docs/quick-start-guide). For very large tables, the same
pattern can be applied partition [...]
+
+If writers must keep running during a long rewrite, capture the cut point (a
Delta version), rewrite up to it, then apply the trailing changes to the Hudi
table before cutover — or accept a short write pause and skip the
reconciliation entirely.
+
+## Handling Delta-Specific Features Honestly
+
+Metadata translation is possible because both formats describe Parquet files —
but the formats are not feature-identical, and the mapping has edge cases. Per
XTable's documented [features and
limitations](https://xtable.apache.org/docs/features-and-limitations), audit
each table for the following before choosing Option A:
+
+- **Deletion vectors.** XTable currently syncs Copy-on-Write / read-optimized
views of tables; Delta deletion vectors are *not* captured by the sync. If a
Delta table uses deletion vectors, a converted Hudi view could include deleted
rows. Purge deletion vectors on the Delta side first (rewriting affected files
so deletes are physically applied), or use the full-rewrite path for those
tables.
+- **Generated columns.** Generated columns on a Delta source do not carry over
to the target schema, and partitioning on generated columns has restricted
support (common date transformations, such as deriving a date partition from a
timestamp, are handled). Tables that partition on other generated expressions
need per-table verification or a rewrite.
+- **Column mapping.** Delta's column mapping decouples logical column names
from the physical names inside Parquet files. Since a converted table reads the
same physical files, tables with column mapping enabled (typically after column
renames or drops) deserve explicit schema validation on the Hudi side before
you rely on them.
+
+The general rule: XTable is faithful for the mainstream case — Parquet data
files, identity or date-derived partitioning, no unapplied deletion vectors —
and conservative engineering means *validating each table* rather than assuming
the mainstream case. The validation checklist below is not optional garnish; it
is how you catch the exceptions.
+
+## Validation Checklist
+
+Run this per table, during the side-by-side period (Option A) or after the
rewrite (Option B), before any writer cutover or consumer switch:
+
+1. **Row counts.** `SELECT COUNT(*)` through the Delta path and the Hudi path
must match at the same sync point.
+2. **Checksums on sample partitions.** Compare aggregate fingerprints — sums,
min/max of key columns, or a hash aggregate — on a handful of partitions,
including at least one recently written partition and one old one.
+3. **Schema comparison.** Diff the schemas reported by both formats, paying
attention to nullability, nested fields, and any renamed columns (see column
mapping above).
+4. **Query-engine smoke tests.** Run representative queries through every
engine that will read the Hudi table — Spark, Trino, Presto, Athena, etc. —
including partition-pruned queries and, if relevant, time travel and
incremental queries.
+5. **Catalog re-registration.** Register the Hudi table in your catalog and
confirm downstream tools resolve it: BI connections, dbt sources,
permissions/grants, and any data-quality jobs pointing at the catalog entry.
+6. **Write-path rehearsal.** Before the real cutover, run the intended Hudi
writer against a cloned or staging copy and confirm upserts, deletes, and table
services behave as expected with your chosen keys and configs.
+
+## Rollback: The Source Table Stays Intact
+
+The most underrated property of both migration paths is that they are
non-destructive. With XTable, the Delta transaction log is never modified —
Hudi metadata is written *alongside* it, and both remain readable throughout
the transition. With a full rewrite, the Delta table sits untouched at its
original path. In either case, rollback before cutover is simply "keep using
the Delta table," and rollback shortly after cutover means pointing writers
back at Delta and replaying the handful o [...]
+
+Keep the source Delta table — and its `_delta_log` — until the Hudi table has
run in production long enough to cover your validation and audit horizon. Only
then decommission it. Migrations fail safe when deletion is the last step, not
a side effect.
+
+## Conclusion
+
+The strongest reason this migration is more approachable than it used to be is
that it is no longer a leap of faith. Because Delta Lake and Hudi both store
data as Parquet, Apache XTable turns "migrate" into "add a second metadata
layer and evaluate" — you can read your existing tables as Hudi today, run both
formats side by side for weeks, validate every engine and consumer, and cut
writers over only when the evidence says to. And if the evidence says
otherwise, the Delta table never st [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'Do I have to rewrite my data to move from Delta Lake to Hudi?',
answer: 'No. Because both formats store data as Apache Parquet, Apache XTable
can translate the Delta transaction log into Hudi metadata alongside the same
data files, with no data copying. A full rewrite is only needed if you want to
change the physical layout, such as partitioning or file sizes, or if the table
uses features like deletion vectors that the metadata sync does not capture.'},
+ {question: 'Can I run Delta Lake and Hudi side by side on the same table?',
answer: 'Yes. XTable writes Hudi metadata next to the existing Delta log, so
the same directory is readable as both a Delta table and a Hudi table. You can
keep the Delta writer running and re-run XTable incrementally to keep both
metadata layers in sync while you validate the Hudi side, which is the
recommended de-risking approach before any writer cutover.'},
+ {question: 'Will my Databricks jobs still work after migrating to Hudi?',
answer: 'Read-only jobs can keep working if you sync the Hudi table back to
Delta with XTable, since Databricks then reads familiar Delta metadata. Jobs
that write to the table must be moved to Hudi writers, and Databricks-specific
features tied to Delta, such as deletion vectors or certain Unity Catalog
integrations, do not carry over. Treat every Databricks job as something to
test explicitly during the side-by [...]
+ {question: 'How long does an XTable conversion take?', answer: 'The sync
reads and writes table metadata only, so its runtime scales with the number of
files and commits rather than with data volume. That makes it dramatically
faster than rewriting the data, and incremental sync keeps subsequent runs
short by translating only new commits.'},
+ {question: 'What happens to my Delta table history and time travel?',
answer: 'The Delta transaction log is left untouched, so Delta-side history
remains fully intact and queryable until you decommission the table. The Hudi
table maintains its own timeline going forward, and it is prudent to keep the
original Delta log around through your audit and validation horizon before
deleting anything.'},
+ {question: 'What if my Delta table uses deletion vectors?', answer: 'XTable
currently syncs Copy-on-Write or read-optimized views, and Delta deletion
vectors are not captured by the sync, so a converted view could expose deleted
rows. Purge the deletion vectors on the Delta side first so deletes are
physically applied to the Parquet files, or migrate that table with a full
Spark rewrite instead.'},
+]} />
diff --git a/website/blog/2026-08-05-migrating-from-apache-iceberg-to-hudi.md
b/website/blog/2026-08-05-migrating-from-apache-iceberg-to-hudi.md
new file mode 100644
index 000000000000..1b626ae7d035
--- /dev/null
+++ b/website/blog/2026-08-05-migrating-from-apache-iceberg-to-hudi.md
@@ -0,0 +1,139 @@
+---
+title: "Migrating from Apache Iceberg to Apache Hudi"
+excerpt: "A practical guide to adopting Apache Hudi on existing Iceberg tables
— via XTable metadata translation or a one-time rewrite — while keeping every
Iceberg-based reader working through reverse sync."
+description: "How to migrate Iceberg tables to Apache Hudi with XTable
metadata conversion or a Spark rewrite, keeping Snowflake, BigQuery and Trino
Iceberg readers working."
+authors: [sivabalan]
+category: how-to
+image:
/assets/images/blog/2025-07-02-Lakehouse-Architecture-apache-hudi-and-apache-iceberg.png
+tags:
+- migration
+- apache xtable
+- apache iceberg
+- guide
+---
+
+You can adopt Apache Hudi on an existing Apache Iceberg table either by
translating its metadata with [Apache XTable](https://xtable.apache.org) — no
data rewrite, since both formats store data as Parquet files — or via a
one-time rewrite; and because XTable also works in the reverse direction,
projecting a Hudi table back out as Iceberg, your existing Iceberg readers can
keep working after the switch. That second point changes the shape of the whole
exercise. "Migrating from Iceberg to [...]
+
+This guide walks through both migration options with working configuration and
code, the cutover sequence that de-risks the switch, an honest look at which
Iceberg features do not map one-to-one, and a validation checklist with a
rollback story. It is the Iceberg-side companion to our guides on [using Hudi
with Apache Iceberg via
XTable](/blog/2026/07/28/using-hudi-with-apache-iceberg-via-xtable) and
[migrating from Delta Lake to
Hudi](/blog/2026/08/04/migrating-from-delta-lake-to-hudi).
+
+## Why Teams Move from Iceberg to Hudi
+
+The migrations we see are almost always driven by the write side. Apache
Iceberg has broad catalog and engine support, and for append-mostly batch
analytics it serves many teams fine. The friction shows up when workloads
become mutation-heavy or latency-sensitive:
+
+- **Record-level indexes for fast upserts.** Hudi maintains a [multi-modal
indexing subsystem](/docs/indexes) — record-level index, bloom filters,
expression and secondary indexes — that maps record keys to file groups. An
upsert locates exactly the files it must touch instead of planning a join or
scan against the target to find matching rows. For CDC pipelines applying
millions of scattered updates, this is routinely the difference between minutes
and hours.
+- **Merge-on-Read designed for streaming ingest.** Hudi's MOR tables absorb
updates as compact log files merged on read, so writers sustain high-frequency
commits — minute-level or faster from Kafka, Flink or Spark Structured
Streaming — without churning out rewritten Parquet on every batch.
+- **Built-in table services.** Compaction, clustering, cleaning and file
sizing are part of the Hudi runtime and run inline or asynchronously without
external orchestration. With Iceberg, that maintenance is left to engines,
scheduled Spark procedures or a vendor service — someone has to own it.
+- **CDC-grade change streams.** Hudi tables serve [incremental
queries](/docs/sql_queries#incremental-query): give me exactly the records that
changed between two points on the timeline, including before/after images in
CDC mode. Downstream pipelines chain off tables directly instead of re-reading
snapshots and diffing.
+
+The gap is measurable. In benchmarks run with the open-source
[LakeLoader](https://github.com/onehouseinc/lake-loader) framework — Spark 3.5,
Hudi 1.1.1, Iceberg 1.10.0, on S3 — we observed roughly 4× lower incremental
write latency for Hudi on a 10 TB partitioned fact table with skewed updates,
and about 8× lower steady-state latency on Merge-on-Read tables taking sparse,
column-level updates, with Hudi's record-level index sidestepping the
full-table-scan merge joins that dominate the [...]
+
+The point is not that Iceberg cannot handle these workloads — it is that
Hudi's write path, indexing and self-managing services are built for mutable,
streaming-oriented workloads. If that describes the tables you are running,
here is how to move them.
+
+## Migrate the Writer, Keep the Readers
+
+The biggest source of migration risk is rarely the table itself — it is the
long tail of consumers. A warehouse reading through an Iceberg catalog, BI
dashboards, other teams' Spark jobs. A migration plan that requires all of them
to change on cutover day is fragile enough that most teams never start.
+
+The two-way XTable pattern removes that requirement:
+
+1. **Writers move to Hudi.** Your ingestion pipeline gains Hudi's upsert
indexes, MOR streaming writes and table services.
+2. **XTable continuously projects the Hudi table back out as Iceberg.** After
each Hudi commit (or on a schedule), XTable translates the Hudi timeline into
Iceberg metadata over the *same* Parquet data files, and its catalog sync can
keep Hive Metastore or AWS Glue entries current.
+3. **Readers keep reading Iceberg.** Snowflake, BigQuery, Trino Iceberg
catalogs, anything else that only speaks Iceberg — all keep working. They can
each move to native Hudi reads later, on their own schedule, or never.
+
+Because the data files are shared and only lightweight metadata is generated,
the reverse projection is cheap and stays fresh. The migration decision
decomposes: the writer cutover is one contained change, and every reader
migration becomes optional and independent. We cover the reader-side mechanics
in depth in the [Hudi + Iceberg interoperability
guide](/blog/2026/07/28/using-hudi-with-apache-iceberg-via-xtable); the rest of
this post focuses on getting the table and the writer onto Hudi.
+
+## Option A: Convert Iceberg Metadata to Hudi with Apache XTable
+
+XTable (incubating) translates table metadata between Hudi, Iceberg and Delta
Lake in any direction. Pointed at an Iceberg table, it reads the Iceberg
snapshot and writes Hudi metadata — a `.hoodie` timeline with schema, commit
history, partition and column statistics — referencing the existing Parquet
files in place. Nothing is copied or rewritten.
+
+Grab the XTable bundled jar (build from
[source](https://github.com/apache/incubator-xtable) or download from GitHub
packages) and create a config:
+
+```yaml md title="iceberg_to_hudi.yaml"
+sourceFormat: ICEBERG
+targetFormats:
+ - HUDI
+datasets:
+ -
+ tableBasePath: s3://warehouse/orders
+ tableDataPath: s3://warehouse/orders/data
+ tableName: orders
+ partitionSpec: order_date:VALUE
+```
+
+`tableDataPath` is needed for Iceberg sources when data files live under a
subdirectory (the layout Iceberg warehouses typically use) rather than directly
under the base path. Then run the sync:
+
+```shell
+java -jar path/to/xtable-utilities-bundled.jar --datasetConfig
iceberg_to_hudi.yaml
+```
+
+The run produces Hudi metadata under the table's base path. Any Hudi-capable
engine can now read the table — the same files your Iceberg readers are still
using. XTable syncs incrementally by default (translating only new commits,
falling back to a full sync when needed), so re-running it on a schedule keeps
the Hudi view current for as long as the Iceberg writer remains active.
Registering the table in your catalog of choice makes it visible to Hudi
readers alongside the existing Iceber [...]
+
+At this stage you have a zero-copy, read-ready Hudi table and have changed
nothing about production. That alone is useful — you can benchmark Hudi readers
against real data before committing to anything.
+
+## Converted vs Native: Cutting the Writer Over
+
+Metadata translation gets you Hudi *reads*. The features that motivate the
migration — record-level index lookups, streaming upserts, change streams,
self-managing table services — come from Hudi *writing* the table: assigning
record keys, maintaining indexes and the metadata table, running services
against its own timeline. To get them, you cut the writer over. The sequence:
+
+1. **Stop the Iceberg writer.** Pause ingestion at a clean commit boundary.
+2. **Run a final XTable sync** (Iceberg → Hudi) so the Hudi metadata reflects
the last Iceberg commit exactly.
+3. **Start the Hudi writer** against the table's base path, configuring the
record key, ordering fields (`hoodie.table.ordering.fields`) and partitioning
to match your workload. From here, commits land natively on the Hudi timeline,
indexes are built and maintained, and table services take over maintenance.
+4. **Enable reverse sync** (Hudi → Iceberg) — via the XTable job on a schedule
with `sourceFormat: HUDI`, or per-commit through the Hudi Streamer sync
extension ([docs](/docs/syncing_xtable)) — so Iceberg readers see every new
Hudi commit.
+
+The gap between steps 1 and 4 is minutes of paused ingestion, not a data-copy
window — for most tables the downtime is a single deferred micro-batch.
Validate the native-write onboarding on a staging copy first: depending on the
table's layout and key structure, some tables are better served by Hudi's
[bootstrap mechanism](/docs/migration_guide) or a full rewrite (Option B below)
to get a fully native file layout with populated Hudi metadata fields, rather
than writing directly on top of [...]
+
+## Option B: One-Time Rewrite with Spark
+
+If the table is modest in size, or you want to change its physical layout
anyway — new partitioning, clustering by query predicates, cleaning out
accumulated small files — a full rewrite is the simpler and sometimes better
move. Read the Iceberg table with Spark, write a Hudi table:
+
+```scala
+// spark-shell with both Iceberg and Hudi bundles on the classpath
+val df = spark.read.format("iceberg").load("s3://warehouse/orders")
+
+df.write.format("hudi").
+ option("hoodie.table.name", "orders").
+ option("hoodie.datasource.write.recordkey.field", "order_id").
+ option("hoodie.datasource.write.partitionpath.field", "order_date").
+ option("hoodie.table.ordering.fields", "updated_at").
+ option("hoodie.datasource.write.operation", "bulk_insert").
+ mode("overwrite").
+ save("s3://warehouse/orders_hudi")
+```
+
+This is a plain batch job — parallelize by partition for very large tables, as
shown in the [migration guide](/docs/migration_guide), which also covers Hudi
Streamer's bootstrap mode (including METADATA_ONLY, which builds skeleton files
instead of rewriting data). The rewrite costs a full pass over the data but
produces a completely native Hudi table with no conversion caveats, and cutover
is just repointing the writer and backfilling the delta that accrued during the
copy. See the [Spar [...]
+
+## What May Not Map One-to-One
+
+An honest migration plan checks feature parity per table rather than assuming
it. Watch for:
+
+- **Merge-on-read delete files.** XTable syncs the underlying Parquet data
files; Iceberg v2 position/equality delete files are not carried through the
conversion. Compact them away first (e.g. Iceberg's `rewrite_data_files` /
`rewrite_position_delete_files` procedures) so the source snapshot is
materialized in data files before converting. The same applies to newer v3
constructs such as deletion vectors.
+- **Hidden partitioning and partition evolution.** Iceberg partitions by
column transforms (`days(ts)`, `bucket(n, id)`) invisible to queries, and lets
the partition spec evolve over time. Hudi partitions by explicit partition path
fields. Simple value and date-based partitioning translate cleanly; bucket
transforms and tables carrying multiple historical partition specs need
per-table validation, and a rewrite (Option B) into a layout of your choosing
is often the cleaner answer for them.
+- **Iceberg-specific column types and metadata.** Features tied to recent
Iceberg spec versions — row lineage, variant type in v3, engine-specific
catalog behaviors — have no direct Hudi equivalent or map differently.
Inventory what each table actually uses; most analytics tables use none of
these.
+- **Snapshot history.** The converted Hudi table's timeline starts at
conversion. Old Iceberg snapshots remain time-travelable through the Iceberg
metadata (which stays intact on storage), but do not appear as Hudi commits.
+
+None of these block a migration; they determine which tables take the
metadata-translation fast path and which deserve a rewrite.
+
+## Validation Checklist and Rollback
+
+Before decommissioning anything, verify per table:
+
+- **Row counts and checksums** match between the Iceberg source and Hudi
target (and, after cutover, between the Hudi table and its reverse-synced
Iceberg projection).
+- **Schema fidelity** — column types, nullability and nested structures
survived translation.
+- **Partition pruning** works: run a partition-filtered query against the Hudi
table and confirm file skipping.
+- **Reader smoke tests** from every consumer that matters: native Hudi reads
from Spark/Trino, and Iceberg reads from the warehouse or catalog your
downstream teams use.
+- **Writer dry run** on staging: upsert, delete, then read back and confirm
index behavior and table service activity on the timeline.
+
+The rollback story is what makes this migration low-stakes. Until you delete
it, the original Iceberg metadata is untouched — XTable conversion writes new
metadata alongside, and Option B writes to a new path entirely — so reverting
before cutover means simply not proceeding. After cutover, reverse sync means
there is a live, continuously updated Iceberg view of the table at all times:
if a downstream Iceberg consumer misbehaves, it keeps reading Iceberg while you
debug, and in the worst [...]
+
+## Conclusion
+
+Table format choice used to be a one-way architectural door; XTable makes it a
revolving one. Adopting Hudi on an Iceberg estate is not a monolithic rewrite:
translate metadata in place to light up Hudi reads, cut writers over
table-by-table to gain [indexes](/docs/indexes), streaming upserts and built-in
table services, and let reverse sync keep every Iceberg-based consumer running
exactly as before. Rewrite the tables that want a new layout; translate the
rest. Start with one mutation- [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'Can Snowflake still read my table after migrating to Hudi?',
answer: 'Yes. Apache XTable can continuously project a Hudi table back out as
Iceberg metadata over the same Parquet data files, and can sync catalogs like
AWS Glue or Hive Metastore. Snowflake, BigQuery and other engines that read
Iceberg keep working against that projection, so reader migrations become
optional and can happen on their own schedule.'},
+ {question: 'Does migrating from Iceberg to Hudi require rewriting data?',
answer: 'Not necessarily. Both formats store data as Parquet files, so Apache
XTable can translate the Iceberg metadata into a Hudi timeline referencing the
existing files in place, with no data copy. A full rewrite via Spark is only
needed when you want to change the physical layout, such as new partitioning or
clustering, or for tables whose Iceberg features don\'t translate cleanly.'},
+ {question: 'What do I gain by moving the write path to Hudi?', answer:
'Hudi\'s writer maintains record-level and bloom indexes that make scattered
upserts fast, supports Merge-on-Read tables built for high-frequency streaming
ingestion, runs compaction, clustering and cleaning as built-in table services,
and serves incremental and CDC queries off its timeline. These come from Hudi
writing the table, which is why the cutover targets writers first.'},
+ {question: 'Can I go back to Iceberg if the migration doesn\'t work out?',
answer: 'Yes. The original Iceberg metadata stays intact on storage until you
delete it, so before cutover, rollback means simply not proceeding. After
cutover, reverse sync maintains a live Iceberg view of the table at all times,
so in the worst case you can resume Iceberg-native writes from the current
state rather than restoring from backups.'},
+ {question: 'How much downtime does the writer cutover take?', answer:
'Minutes, not hours. The sequence is: pause the Iceberg writer at a commit
boundary, run a final XTable sync so the Hudi metadata matches exactly, start
the Hudi writer, and enable reverse sync from Hudi back to Iceberg. There is no
data copy in that window, so the downtime is typically a single deferred
micro-batch.'},
+ {question: 'Do Iceberg v2 delete files survive the conversion?', answer:
'No. XTable syncs the underlying Parquet data files, and Iceberg position or
equality delete files are not carried through. Run Iceberg\'s compaction
procedures to materialize deletes into data files before converting, so the
snapshot XTable translates is fully represented in Parquet.'},
+]} />
diff --git a/website/blog/2026-08-06-hudi-vs-iceberg-for-cdc-workloads.md
b/website/blog/2026-08-06-hudi-vs-iceberg-for-cdc-workloads.md
new file mode 100644
index 000000000000..389586c2462d
--- /dev/null
+++ b/website/blog/2026-08-06-hudi-vs-iceberg-for-cdc-workloads.md
@@ -0,0 +1,110 @@
+---
+title: "Apache Hudi vs Apache Iceberg for CDC Workloads"
+excerpt: "A mechanism-level comparison of how Apache Hudi and Apache Iceberg
handle change data capture: record lookup, deletes, small files, ordering and
change streams."
+description: "Hudi vs Iceberg for CDC: how each handles record lookups,
deletes, small files, ordering and change streams — and a framework for
choosing between them."
+authors: [sivabalan]
+category: deep-dive
+image: /assets/images/blog/2024-12-03-apache-iceberg-vs-apache-hudi.jpeg
+tags:
+- comparison
+- apache iceberg
+- cdc
+- data lakehouse
+---
+
+Both Apache Hudi and Apache Iceberg can ingest change data capture (CDC)
streams, but they were designed around different problems. Hudi was built for
mutation-heavy ingestion: a record-level index that locates any key without
scanning the table, merge-on-read delta logs that absorb frequent updates and
deletes cheaply, and built-in deduplication, ordering and file sizing. Iceberg
was designed first for large, reliable analytical scans; its copy-on-write and
delete-file-based merge-on-re [...]
+
+<!--truncate-->
+
+## What CDC demands from a table format
+
+A CDC stream is, by construction, mostly mutations. If you replicate an
operational database to the lake (the basics are covered in [What is CDC on a
Data Lake?](/blog/2026/07/22/what-is-cdc-on-a-data-lake) and the deeper
[Understanding Data Lake Change Data Capture](/blog/2024/07/30/data-lake-cdc)),
the table format underneath has to sustain a specific set of pressures:
+
+- **Fast key-based upserts and deletes.** Every change event targets one row
identified by a primary key. The write path must find where that key lives and
apply the change without rewriting or rescanning unrelated data.
+- **Ordering and late data.** Events arrive out of order after retries and
repartitioning. The format must merge by the source's log position or event
time, not arrival time, or the table silently regresses.
+- **Frequent small commits.** Minute-level freshness means committing every
few minutes, around the clock. That stresses metadata, file counts and any
background maintenance.
+- **Change streams for downstream consumers.** Bronze tables that absorb CDC
should also *emit* changes, so silver and gold tables can be built
incrementally instead of re-reading everything.
+
+Hold each format against these four demands and the architectural differences
become concrete.
+
+## Finding the record: indexes vs. joins
+
+The first job in applying an upsert is answering "which file contains this
key?"
+
+**Hudi** treats indexing as a core component of the write path. Every record
key maps to exactly one file group, and Hudi maintains [pluggable
indexes](/docs/indexes) — bloom-filter, simple-join, bucket-hash and, since
0.14.0, a record-level index stored inside the table's metadata table — that
resolve incoming keys to their file groups directly. With the record-level
index, lookup cost is proportional to the number of records changed, not the
size of the table: the writer consults hash- [...]
+
+**Iceberg** deliberately has no record-level index; its metadata tree
(manifests, partition values, column min/max stats) is built for pruning scans,
not locating keys. A `MERGE INTO` in Spark plans a join between the incoming
batch and the target table to discover which files hold matching rows — column
statistics can prune files, but for keys spread across a table the merge
effectively scans and joins against much of it. Streaming writers such as
Flink's Iceberg sink in upsert mode sid [...]
+
+This difference is measurable, and you can reproduce it. In benchmarks run
with the open-source [LakeLoader
framework](https://github.com/onehouseinc/lake-loader) (Spark 3.5, Hudi 1.1.1,
Iceberg 1.10.0), we observed Hudi delivering roughly 4× lower incremental write
latency than Iceberg on a 10 TB partitioned table under skewed CDC-style
updates — the record-level index shuffled around 250 MB per commit where the
merge join shuffled hundreds of gigabytes — and roughly 8× lower steady-sta
[...]
+
+## How deletes actually work
+
+Deletes are where CDC pipelines live or die, since every update in a changelog
is logically a delete plus an insert of the new version.
+
+**Hudi** logs a delete into the same file group where the key lives, exactly
like an update. On a merge-on-read table this is an append to that file group's
delta log; queries merge base file plus log at read time within the file group,
and compaction later rewrites the base file with the deletes applied. Because
deletes are colocated with the data they affect, the merge scope is always one
base file against its own changes — never a table-wide reconciliation. Data
locality is preserved [...]
+
+**Iceberg** (format v2) represents row-level deletes as separate delete files
of two kinds. *Position deletes* name a row by data file path and row ordinal —
precise and cheap to apply on read, but the writer must first find the row,
which without an index means the join described above. *Equality deletes*
record column predicates and are cheap to write, but every reader must check
each equality delete file against every data file with an older sequence number
in its scan. When many data [...]
+
+The Iceberg community has been working on this. Format v3 replaces position
delete files with binary *deletion vectors* — at most one compact bitmap per
data file — which removes much of the position-delete proliferation and
read-time association cost, and engines are adopting it. But deletion vectors
help the position-delete path; a writer still needs to determine which
positions to delete, so the no-index lookup cost remains, and equality deletes
still defer their cost to readers and c [...]
+
+## Commit frequency and small files
+
+CDC means committing every few minutes, forever. Two things degrade under that
regime: file sizes and metadata.
+
+**Hudi** performs automatic file sizing during writes — small inserts are
bin-packed into existing under-sized file groups, so frequent commits do not
proliferate small files in the first place. Its cleaner bounds how many old
file versions are retained, and timeline archival keeps the active timeline
small, all running as managed table services alongside ingestion.
+
+**Iceberg** produces at least one new data file (and, in merge-on-read, delete
files) plus a new snapshot per commit. Nothing in the write path resizes files,
so a minute-level stream steadily accumulates small data files, delete files
and snapshot metadata. The remedies exist — `rewrite_data_files` for
compaction, `expire_snapshots`, `rewrite_manifests`,
`rewrite_position_delete_files` — but they are maintenance procedures you
schedule and size yourself, and under optimistic concurrency [...]
+
+## Ordering, dedup and partial updates
+
+Changelogs carry semantics beyond "latest write wins."
+
+**Hudi** requires a record key and supports an ordering (precombine) field as
first-class table concepts. Multiple events for the same key within a batch are
deduplicated by ordering value before write; a late-arriving event with an
older ordering value than what's stored is dropped rather than applied, so
out-of-order delivery cannot regress the table. Record merger and payload APIs
go further: partial updates merge only the non-null incoming fields into the
existing row, and custom mer [...]
+
+**Iceberg** does not define keys, dedup or ordered merging at the format level
— it stores what engines write. Spark's `MERGE INTO` will fail if multiple
source rows match one target row, so deduplication and event-time ordering must
be implemented upstream in the pipeline; Flink's upsert mode assumes the stream
itself is correctly ordered per key within its checkpoint discipline. That is a
deliberate design choice — the format stays engine-neutral — but it means every
CDC pipeline re-im [...]
+
+## Producing change streams downstream
+
+A CDC-fed bronze table is usually the head of a pipeline, so how each format
*emits* changes matters as much as how it absorbs them.
+
+**Hudi** keeps commit-time metadata on every record and supports [incremental
queries](/docs/table_types) as a native query type: give a begin instant, get
exactly the records that changed since, at any commit granularity. Since 0.13.0
a CDC query mode also returns database-style before/after images with the
operation type, logged at write time. Downstream jobs chain these into
incremental ETL, each layer consuming only deltas.
+
+**Iceberg** supports incremental scans between snapshots, which cover
append-only snapshots; consuming row-level changes from snapshots that carry
updates and deletes is harder, and Spark's `create_changelog_view` procedure
computes a changelog view by diffing snapshots — with pre/post images available
as an optional, more expensive computation — rather than reading changes that
were logged at write time. For append-mostly tables the two are comparable; for
update-heavy tables, deriving [...]
+
+## Operational reality
+
+What do you actually run in production?
+
+A Hudi ingestion job carries its table services with it: compaction,
clustering, cleaning, file sizing and archival are scheduled and executed by
the platform, inline or async, with [MVCC coordinating writers and table
services](/docs/concurrency_control) so compaction does not block ingestion —
plus non-blocking concurrency control (since 1.0) for multiple writers, and
early conflict detection that aborts a doomed write mid-flight instead of at
commit, saving the wasted compute. The ing [...]
+
+An Iceberg deployment composes the equivalent from parts: an orchestrator (or
a managed catalog service) running compaction, snapshot expiry, delete-file
rewrites and manifest rewrites at the right cadence for your commit rate, with
conflict-retry behavior tuned so maintenance and streaming writers coexist.
None of this is exotic, and managed platforms increasingly automate it — but on
a high-frequency CDC table the maintenance cadence is unforgiving, and it is
your pager when it falls behind.
+
+## A decision framework
+
+Honest summary rather than a scorecard:
+
+**Iceberg is a workable choice when** your CDC is low-frequency (hourly or
slower batches), tables are append-mostly with modest update ratios, you
already operate an Iceberg estate with maintenance automation in place, or
organizational standardization on one format outweighs per-workload
optimization. The v3 deletion-vector work also means the read-side delete
overhead is shrinking release by release.
+
+**Hudi pulls ahead when** update/delete ratios are high (mirroring OLTP
tables, not appending events), you need sub-hour freshness with commits every
few minutes, keys are spread across the table so per-write lookup cost
dominates, or downstream consumers need true record-level change streams. These
are exactly the pressures Hudi's index, delta logs, precombine semantics and
managed services were designed against.
+
+## It's Not Either/Or: Interoperability via Apache XTable
+
+The framing of this post — pick one format — understates a practical third
option. The formats' metadata layers describe data files; the data files
themselves are Parquet either way. [Apache XTable](https://xtable.apache.org/)
(incubating) exploits this by translating table metadata between Hudi, Iceberg
and Delta Lake without copying data, so a single physical table can be read
through more than one format's metadata.
+
+For CDC specifically, that enables a concrete pattern: ingest with Hudi to get
the record-level index, log-based deletes, auto file sizing and incremental
queries on the write-heavy bronze layer, then run XTable to expose the same
data as Iceberg metadata for engines and catalogs that only speak Iceberg —
Snowflake, BigQuery, or an existing Iceberg-standardized analytics stack. The
translation is incremental (it processes only new commits) and involves no data
rewrite, since both formats [...]
+
+This decouples the write-path decision from the read-path decision. Choose the
ingestion machinery on the merits of your CDC workload, and serve readers in
whatever format they require. We cover the setup end-to-end in [Using Hudi with
Apache Iceberg via
XTable](/blog/2026/07/28/using-hudi-with-apache-iceberg-via-xtable).
+
+## Conclusion
+
+Hudi and Iceberg will both land a CDC stream. The difference is where the cost
of mutation is paid. Hudi pays it at write time with machinery built for the
purpose — an index that finds keys in O(changes), delta logs that keep deletes
local to their file group, ordering and dedup in the format, file sizing and
compaction as built-in services. Iceberg defers it — to read-time delete
reconciliation, to maintenance jobs, to pipeline-level dedup logic — a trade
that suits scan-heavy, append- [...]
+
+## FAQ
+
+<PostFAQ heading={null} items={[
+ {question: 'Is Apache Iceberg good for CDC?', answer: 'Iceberg can handle
CDC, and it works well when changes arrive in low-frequency batches or tables
are append-mostly. For high-frequency, update-heavy streams, the lack of a
record-level index makes merges join against the table, delete files accumulate
between compactions, and you must schedule maintenance jobs aggressively.
Iceberg v3\'s deletion vectors improve the read-side cost, but the write-side
lookup and maintenance burden r [...]
+ {question: 'Why is Hudi faster for upserts than Iceberg?', answer: 'Hudi
maintains indexes, including a record-level index, that map each record key
directly to the file group holding it, so an upsert touches only the files that
actually contain changed keys and lookup cost scales with the size of the
change batch. Iceberg has no record index, so finding the rows to update
requires joining the incoming batch against the target table. On merge-on-read
tables Hudi also appends changes to [...]
+ {question: 'How do deletes differ between Hudi and Iceberg?', answer: 'Hudi
logs a delete into the same file group where the record lives, and compaction
later applies it while rewriting that one base file, so merge scope stays
local. Iceberg v2 writes separate position or equality delete files that
readers and compaction jobs must reconcile against data files, a cost that
grows as both accumulate. Iceberg v3 replaces position delete files with
per-file deletion vectors, which meaningf [...]
+ {question: 'Can I use Hudi for CDC ingestion and still serve Iceberg
readers?', answer: 'Yes. Apache XTable translates Hudi table metadata into
Iceberg metadata incrementally, without copying or rewriting the underlying
Parquet data files. You get Hudi\'s indexed upserts, delete handling and file
sizing on the write path, while Iceberg-only engines and catalogs read the same
table through Iceberg metadata.'},
+ {question: 'Does Iceberg support incremental reads like Hudi?', answer:
'Partially. Iceberg supports incremental scans over append snapshots, and Spark
can build a changelog view by diffing snapshots, with before/after images as an
extra computation. Hudi records commit metadata on every record and logs change
data at write time, so incremental and CDC queries read changes directly
instead of deriving them, which matters most on update-heavy tables.'},
+]} />
diff --git a/website/docs/migration_guide.md b/website/docs/migration_guide.md
index 513a201e6672..831f0d7c74a1 100644
--- a/website/docs/migration_guide.md
+++ b/website/docs/migration_guide.md
@@ -121,6 +121,13 @@ Here are the basic configs that control bootstrapping.
By default, with only `hoodie.bootstrap.base.path` being provided
METADATA_ONLY mode is selected. For other options, please refer [bootstrap
configs](https://hudi.apache.org/docs/next/configurations#Bootstrap-Configs)
for more details.
## Related Resources
+<h3>Step-by-step migration guides</h3>
+
+* [Migrating from Parquet to Apache
Hudi](https://hudi.apache.org/blog/2026/07/29/migrating-from-parquet-to-hudi)
+* [Migrating from Apache Hive Tables to Apache
Hudi](https://hudi.apache.org/blog/2026/07/30/migrating-from-hive-to-hudi)
+* [Migrating from Delta Lake to Apache
Hudi](https://hudi.apache.org/blog/2026/08/04/migrating-from-delta-lake-to-hudi)
+* [Migrating from Apache Iceberg to Apache
Hudi](https://hudi.apache.org/blog/2026/08/05/migrating-from-apache-iceberg-to-hudi)
+
<h3>Videos</h3>
* [Bootstrapping in Apache Hudi on EMR Serverless with
Lab](https://www.youtube.com/watch?v=iTNLqbW3YYA)
diff --git a/website/static/llms.txt b/website/static/llms.txt
index 5177506057f8..f8fbfb4df255 100644
--- a/website/static/llms.txt
+++ b/website/static/llms.txt
@@ -53,9 +53,12 @@ It can also synchronize your data to half dozen data
catalogs to keep table cons
- [Migrating from Parquet to
Hudi](https://hudi.apache.org/blog/2026/07/29/migrating-from-parquet-to-hudi):
In-place bootstrap vs full rewrite for converting raw Parquet datasets into
Hudi tables.
- [Migrating from Hive to
Hudi](https://hudi.apache.org/blog/2026/07/30/migrating-from-hive-to-hudi):
Moving Hive tables to Hudi with bootstrap and catalog sync.
+- [Migrating from Delta Lake to
Hudi](https://hudi.apache.org/blog/2026/08/04/migrating-from-delta-lake-to-hudi):
Metadata conversion with Apache XTable or full rewrite, step by step.
+- [Migrating from Apache Iceberg to
Hudi](https://hudi.apache.org/blog/2026/08/05/migrating-from-apache-iceberg-to-hudi):
Keep Iceberg readers working via XTable while adopting Hudi's write path.
## Blog — Comparisons
+- [Hudi vs Iceberg for CDC
Workloads](https://hudi.apache.org/blog/2026/08/06/hudi-vs-iceberg-for-cdc-workloads):
Criteria-driven comparison for change-data-capture ingestion.
- [Hudi vs Delta Lake vs Iceberg Feature
Comparison](https://hudi.apache.org/blog/2023/01/11/Apache-Hudi-vs-Delta-Lake-vs-Apache-Iceberg-Lakehouse-Feature-Comparison):
Feature-by-feature comparison of the three open table formats.
## Blog — Deep dives
