This is an automated email from the ASF dual-hosted git repository.
JingsongLi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/paimon.git
The following commit(s) were added to refs/heads/master by this push:
new fc09fc7404 [docs] Add documentation for Flink lineage support (#8476)
fc09fc7404 is described below
commit fc09fc7404e68b0b5bfd4ce579825a2812828d78
Author: jsingh-yelp <[email protected]>
AuthorDate: Thu Jul 9 01:12:37 2026 -0400
[docs] Add documentation for Flink lineage support (#8476)
---
docs/docs/flink/lineage.md | 38 ++++++++++++++++++++++++++++++++++++++
docs/sidebars.js | 3 ++-
2 files changed, 40 insertions(+), 1 deletion(-)
diff --git a/docs/docs/flink/lineage.md b/docs/docs/flink/lineage.md
new file mode 100644
index 0000000000..8c4c99008f
--- /dev/null
+++ b/docs/docs/flink/lineage.md
@@ -0,0 +1,38 @@
+---
+title: "Data Lineage"
+sidebar_position: 13
+---
+
+# Data Lineage
+
+Paimon implements
[FLIP-314](https://cwiki.apache.org/confluence/spaces/FLINK/pages/255070913/FLIP-314+Support+Customized+Job+Lineage+Listener)
to expose lineage information through Flink's native
[LineageVertexProvider](https://nightlies.apache.org/flink/flink-docs-stable/docs/internals/data_lineage/)
interface. This allows lineage consumers (such as
[OpenLineage](https://openlineage.io/docs/integrations/flink/flink2)) to
automatically discover Paimon datasets in the Flink job graph.
+
+:::info
+Data Lineage support is available only for Flink 2.0 and above.
+:::
+
+## Table API / SQL
+
+When using Paimon tables via Flink SQL or Table API, lineage is automatically
reported for both source and sink tables.
+
+Apart from the table lineage information provided by Flink natively (name,
schema), Paimon enriches each dataset with a namespace and a **config facet**
containing core table options and Iceberg metadata options.
+
+:::warning
+Full Table API lineage support requires <a
href="https://github.com/apache/flink/pull/27727"; style="color: #1677ff;
text-decoration: underline;">this upstream Flink change</a> which is pending
merge. Without it, Flink will not call `getLineageVertex()` for
`DataStreamScanProvider` and `DataStreamSinkProvider`, which can lead to
potentially missing Paimon input or output datasets. This applies to all Flink
2.x versions that do not yet include this change.
+:::
+
+## DataStream API
+
+Paimon also supports lineage for the DataStream API. Lineage is automatically
reported for sources built via `FlinkSourceBuilder` and for all sinks that
extend `FlinkSink`.
+
+Similar to the Table API, it reports all lineage information including name,
namespace, schema facet, and config facet.
+
+## Configuration
+
+To consume lineage events, a
[JobStatusChangedListener](https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/advanced/job_status_listener/#configuration)
must be added in your Flink configuration.
+
+For example, to configure with OpenLineage:
+
+```
+execution.job-status-changed-listeners =
io.openlineage.flink.listener.OpenLineageJobStatusChangedListenerFactory
+```
diff --git a/docs/sidebars.js b/docs/sidebars.js
index 62a0f3ec03..4223c1347f 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -151,7 +151,8 @@ const sidebars = {
"flink/default-value",
"flink/procedures",
"flink/action-jars",
- "flink/savepoint"
+ "flink/savepoint",
+ "flink/lineage"
]
},
{
