This is an automated email from the ASF dual-hosted git repository.
Abacn pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new ade938f14f3 Update Beam website roadmap pages (#39899)
ade938f14f3 is described below
commit ade938f14f30cb5d6d9423de807d574329bacc62
Author: Yi Hu <[email protected]>
AuthorDate: Fri Aug 28 12:06:42 2026 -0400
Update Beam website roadmap pages (#39899)
* Update Beam website roadmap pages
* Update overall roadmap highlights:
- Add section for Extended Metadata and SDK-level CDC support
- Highlight SchemaTransform API, Beam YAML, and Managed I/O under
cross-language transforms
- Update supported runtime versions for Python (3.10-3.14) and Java (11,
17, 21, 25)
* Modernize SDK connector roadmaps:
- Java: Add Extended Metadata/CDC, OpenTelemetry trace context
propagation (#33176), and lakehouse format ecosystem (Iceberg, Delta Lake,
Managed I/O); remove stale connector items
- Python: Reframe around Java cross-language SchemaTransforms,
auto-generated transform wrappers, and native Beam ML / AI ecosystem
- Go: Add portable schemas/logical types (#39684), auto cross-language
wrappers, native utility transforms (GroupIntoBatches, Watch), and first-class
I/Os
- Multi-SDK: Mark legacy roadmap as stale
* Update SDK roadmaps
- Add link to SDK language version compatibility matrix
- Add last updated timestamp
* Refresh runner roadmaps:
- Flink: Document Flink 2.x support and DataStream API batch migration
- Spark: Transition from Spark 3 to Spark 4 and Structured Streaming
improvements
- Dataflow & Prism: Update GitHub issue tracker links and resources
* Mark Euphoria DSL as deprecated and scheduled for removal
* Consolidate individual roadmaps, simplify and less specific
---
website/www/site/content/en/roadmap/_index.md | 40 ++++++----
.../site/content/en/roadmap/connectors-go-sdk.md | 27 -------
.../site/content/en/roadmap/connectors-java-sdk.md | 36 ---------
.../content/en/roadmap/connectors-multi-sdk.md | 92 ++++------------------
.../content/en/roadmap/connectors-python-sdk.md | 29 -------
.../www/site/content/en/roadmap/dataflow-runner.md | 9 ++-
website/www/site/content/en/roadmap/euphoria.md | 2 +
.../www/site/content/en/roadmap/flink-runner.md | 20 ++++-
website/www/site/content/en/roadmap/go-sdk.md | 15 +++-
website/www/site/content/en/roadmap/java-sdk.md | 9 ++-
.../www/site/content/en/roadmap/prism-runner.md | 2 +
website/www/site/content/en/roadmap/python-sdk.md | 35 ++++++--
.../www/site/content/en/roadmap/spark-runner.md | 18 +++--
.../layouts/partials/section-menu/en/roadmap.html | 16 ++--
14 files changed, 130 insertions(+), 220 deletions(-)
diff --git a/website/www/site/content/en/roadmap/_index.md
b/website/www/site/content/en/roadmap/_index.md
index e764898d3f8..a9676da1ee5 100644
--- a/website/www/site/content/en/roadmap/_index.md
+++ b/website/www/site/content/en/roadmap/_index.md
@@ -27,10 +27,6 @@ The major components of Beam each have their own roadmap
which you can find
via the menu.
Below are some highlights for the project as a whole.
-## Beam 3
-
-Beam 3 is the planned first major version upgrade. See
https://s.apache.org/beam3-milestones for details.
-
## Portability Framework
Portability is the primary Beam vision: running pipelines authored with _any
SDK_
@@ -41,27 +37,48 @@ runners.
See the details on the [Portability Roadmap](/roadmap/portability/)
-## Cross-language transforms
+### Portable schemas
+
+Schemas allow SDKs and runners to understand
+the structure of user data and unlock relational optimization possibilities.
+Portable schemas enable compatibility between rows in Python and Java.
+A particularly interesting use case is the combination of SQL (implemented in
Java)
+with the Python SDK via Beam's cross-language support.
+Learn more about portable schemas from this [design
doc](https://s.apache.org/beam-schemas).
+
+### Cross-language transforms
As a benefit of the portability effort, we are able to utilize Beam transforms
across SDKs.
Examples include using Java connectors and Beam SQL from Python or Go pipelines
or Beam TFX transforms from Java and Go.
+
+The [SchemaTransform
API](/documentation/sdks/python-custom-multi-language-pipelines-guide) is now
the preferred way to author cross-language transforms, leveraging Beam Schemas
for configuration and data exchange. Implementing SchemaTransforms is also
required to make transforms available to [Beam YAML](/documentation/sdks/yaml)
and the [Managed I/O](/documentation/io/managed-io) API.
For details see [Roadmap for multi-SDK
efforts](/roadmap/connectors-multi-sdk/).
+## Extended Metadata and SDK level CDC (Change Data Capture) support
+
+Beam is introducing **Extended Metadata** to make element values extensible.
This provides native, cross-SDK support for:
+
+* **Change Data Capture (CDC)**: First-class change operations (`INSERT`,
`UPDATE`, `DELETE`) natively consumed by connectors like Iceberg, Delta Lake,
and Spanner.
+* **Lifecycle & Observability**: Allow SDK level Pipeline drain support
(currently as a Dataflow runner feature).
+* **DoFn API Evolution**: Fluent `OutputBuilder` APIs allowing transforms to
selectively set metadata without combinatorial method overloads.
+
+Learn more from the [Beam Element Extended Metadata Design
Doc](https://s.apache.org/beam-element-extended-metadata).
+
## Go SDK
The Go SDK is not actively being developed beyond bugfixes due to lack of
contributors. If you'd like to help move this forward again, see the [Go SDK's
Roadmap](/roadmap/go-sdk)
## Python 3 support
-As of Apache Beam 2.69.0, we support python version from 3.9 uptil Python
3.13. Supporting Python 3.14 is in our roadmap.
+As of Apache Beam 2.75.0, we support python version from 3.10 uptil Python
3.14. Supporting new Python 3 minor version is in our roadmap.
See details on
the [Python SDK's Roadmap](/roadmap/python-sdk/#python-3-support).
## Java support
-As of Beam 2.69.0, we support Java 8, 11, 17, 21, 25. Java 8 support is
deprecated and scheduled for removal in Beam 3.0.0.
+As of Beam 2.75.0, we support Java 11, 17, 21, 25. Certain modules requires
Java 17+ due to upstream dependency constraints.
See details on the [Java SDK's Roadmap](/roadmap/java-sdk).
## SQL
@@ -70,12 +87,3 @@ Beam's SQL module is rapidly maturing to allow users to
author batch and
streaming pipelines using only SQL, but also to allow Beam Java developers
to use SQL in components of their pipeline for added efficiency. See the
[Beam SQL Roadmap](/roadmap/sql/)
-
-## Portable schemas
-
-Schemas allow SDKs and runners to understand
-the structure of user data and unlock relational optimization possibilities.
-Portable schemas enable compatibility between rows in Python and Java.
-A particularly interesting use case is the combination of SQL (implemented in
Java)
-with the Python SDK via Beam's cross-language support.
-Learn more about portable schemas from this
[presentation](https://s.apache.org/portable-schemas-seattle).
diff --git a/website/www/site/content/en/roadmap/connectors-go-sdk.md
b/website/www/site/content/en/roadmap/connectors-go-sdk.md
deleted file mode 100644
index 2b1692d533e..00000000000
--- a/website/www/site/content/en/roadmap/connectors-go-sdk.md
+++ /dev/null
@@ -1,27 +0,0 @@
----
-title: "Connectors - Go SDK"
----
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-
-Roadmap for connectors developed using Go SDK.
-
-* Go SDK plans to utilize currently available Java and Python connectors
-through cross-language transforms feature.
- * KafkaIO via Java - DONE
- * BigQuery via Java - In Progress
- * Beam SQL via Java
-* The Go SDK supports SplittableDoFns for bounded pipelines, so scalable
bounded pipelines are possible.
- * The textio package supports
[ReadSdf](https://pkg.go.dev/github.com/apache/beam/sdks/v2/go/pkg/beam/io/textio#ReadSdf)
and
[ReadAllSdf](https://pkg.go.dev/github.com/apache/beam/sdks/v2/go/pkg/beam/io/textio#ReadAllSdf)
for efficient batch text reads.
- * A general FileIO will be produced to simplify adding new file based
connectors.
diff --git a/website/www/site/content/en/roadmap/connectors-java-sdk.md
b/website/www/site/content/en/roadmap/connectors-java-sdk.md
deleted file mode 100644
index c9b517e9079..00000000000
--- a/website/www/site/content/en/roadmap/connectors-java-sdk.md
+++ /dev/null
@@ -1,36 +0,0 @@
----
-title: "Connectors - Java SDK"
----
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-
-Roadmap for connectors developed using Java SDK.
-
-# Couchbase
-
-Couchbase is a NoSQL document-oriented database. See
-[Issue 18381](https://github.com/apache/beam/issues/18381) for more details on
the
-planned Beam connector for Couchbase.
-
-# InfluxDB
-
-InfluxDB is a database for fast and highly available storage and retrieval
-of time series data. See
[BEAM-2546](https://issues.apache.org/jira/browse/BEAM-2546) for
-more details on the planned Beam connector for InfluxDB.
-
-# Memcached
-
-Memcached is a distributed memory caching system. See
-[BEAM-1678](https://issues.apache.org/jira/browse/BEAM-1678) for more details
on the
-planned Beam connector for Memcached.
diff --git a/website/www/site/content/en/roadmap/connectors-multi-sdk.md
b/website/www/site/content/en/roadmap/connectors-multi-sdk.md
index 3a404b22bec..c78768c4ad4 100644
--- a/website/www/site/content/en/roadmap/connectors-multi-sdk.md
+++ b/website/www/site/content/en/roadmap/connectors-multi-sdk.md
@@ -1,5 +1,7 @@
---
title: "Multi-SDK Connector Efforts"
+aliases:
+ - /roadmap/connectors-java-sdk/
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
@@ -15,88 +17,28 @@ See the License for the specific language governing
permissions and
limitations under the License.
-->
-Connector-related efforts that will benefit multiple SDKs.
+Roadmap for connectors and cross-cutting efforts that benefit multiple SDKs.
-# Splittable DoFn
-Splittable DoFn is the next generation sources framework for Beam that will
-replace current frameworks for developing bounded and unbounded sources.
-Splittable DoFn is being developed along side current Beam portability
-efforts. See [Beam portability framework roadmap](/roadmap/portability/) for
more details.
+_Last updated on Aug 2026._
-# Cross-language transforms
+# Extended Metadata and SDK level CDC (Change Data Capture) support
-_Last updated on May 2020._
+Beam is introducing **Extended Metadata** to make element values extensible
across SDKs. This provides native support for Change Data Capture (CDC)
operations (`INSERT`, `UPDATE`, `DELETE`) consumed by lakehouse and database
connectors, SDK-level pipeline drain, and fluent `OutputBuilder` APIs.
-As an added benefit of Beam portability effort, we are able to utilize Beam
transforms across SDKs. This has many benefits.
+Learn more from the [Beam Element Extended Metadata Design
Doc](https://s.apache.org/beam-element-extended-metadata).
-* Connector sharing across SDKs. For example,
- + Beam pipelines written using Python and Go SDKs will be able to utilize
the vast selection of connectors that are currently implemented for Java SDK.
- + Java SDK will be able to utilize connectors for systems that only offer a
Python API.
- + Go SDK, will be able to utilize connectors currently available for Java
and Python SDKs.
-* Ease of developing and maintaining Beam transforms - in general, with
cross-language transforms, Beam transform authors will be able to implement new
Beam transforms using a
-language of choice and utilize these transforms from other languages reducing
the maintenance and support overheads.
-* [Beam SQL](/documentation/dsls/sql/overview/), that is currently only
available to Java SDK, will become available to Python and Go SDKs.
-* [Beam TFX transforms](https://www.tensorflow.org/tfx/transform/get_started),
that are currently only available to Beam Python SDK pipelines will become
available to Java and Go SDKs.
+# OpenTelemetry Support
-## Completed and Ongoing Efforts
+Beam is adding native [OpenTelemetry](https://opentelemetry.io/) support
across SDKs, runners, and connectors
([#33176](https://github.com/apache/beam/issues/33176)) for distributed
tracing. This leverages Extended Metadata to propagate W3C Trace Context
(`traceparent` and `tracestate`) across transform stages and worker boundaries,
integrates trace header propagation into streaming I/Os (such as Kafka,
Pub/Sub, and Spanner change streams), and enables runner-level trace and log
correlation.
-Many efforts related to cross-language transforms are currently in flux. Some
of the completed and ongoing efforts are given below.
+# IO Connector Ecosystem
-### Cross-language transforms API and expansion service
+Beam is actively expanding the Beam I/O connector portfolio with a focus on
modern data lakehouse formats, native Change Data Capture (CDC), and unified
access patterns:
-Work related to developing/updating the cross-language transforms API for
Java/Python/Go SDKs and work related to cross-language transform expansion
services.
+* **First-Class Lakehouse Connectors**:
+ * **[Apache Iceberg](/documentation/io/built-in/iceberg/)**: Full support
for batch and streaming reads, streaming appends, dynamic destinations, and
changelog CDC reading. Available natively in Java, via cross-language
transforms in Python, in [Beam YAML](/documentation/sdks/yaml/), and via the
[Managed I/O](/documentation/io/managed-io/) API.
+ * **Delta Lake**: Native read capabilities
([#38551](https://github.com/apache/beam/issues/38551)) and batch changelog
(CDC) reading ([#39492](https://github.com/apache/beam/issues/39492)), exposed
through core transforms and the [Managed I/O](/documentation/io/managed-io/)
API.
+* **Managed I/O Expansion**: Exposing more storage systems through the
[Managed I/O API](/documentation/io/managed-io/), which standardizes connector
configurations using Beam Schemas, provides frictionless cross-language access
(Java, Python, YAML), and enables runner-level optimizations.
+* **Native CDC Integration**: Integrating database change streams (such as
Spanner change streams and Debezium-based sources) directly into lakehouse
sinks using Beam's native CDC `ValueKind` metadata.
-* Basic API for Java SDK - completed
-* Basic API for Python SDK - completed
-* Basic API for Go SDK - In progress
-* Basic cross-language transform expansion service for Java and Python SDKs -
completed
-* Artifact staging - mostly completed - [email
thread](https://lists.apache.org/thread.html/6fcee7047f53cf1c0636fb65367ef70842016d57effe2e5795c4137d@%3Cdev.beam.apache.org%3E),
[doc](https://docs.google.com/document/d/1XaiNekAY2sptuQRIXpjGAyaYdSc-wlJ-VKjl04c8N48/edit#heading=h.900gc947qrw8)
-
-### Support for Flink runner
-
-Work related to making cross-language transforms available for Flink runner.
-
-* Basic support for executing cross-language transforms on portable Flink
runner - completed
-
-### Support for Dataflow runner
-
-Work related to making cross-language transforms available for Dataflow runner.
-
-* Basic support for executing cross-language transforms on Dataflow runner
- + This work requires updates to Dataflow service's job submission and job
execution logic. This is currently being developed at Google.
-
-### Support for Direct runner
-
-Work related to making cross-language transforms available on Direct runner
-
-* Basic support for executing cross-language transforms on Pyton Direct runner
- completed
-* Basic support for executing cross-language transforms on Java Direct runner
- Not started
-
-### Connector/transform support
-
-Ongoing and planned work related to making existing connectors/transforms
available to other SDKs through the cross-language transforms framework.
-
-* Java JdbcIO - completed -
[BEAM-10135](https://issues.apache.org/jira/browse/BEAM-10135),
[BEAM-10136](https://issues.apache.org/jira/browse/BEAM-10136)
-* Java KafkaIO - completed -
[BEAM-7029](https://issues.apache.org/jira/browse/BEAM-7029)
-* Java KinesisIO - completed -
[BEAM-10137](https://issues.apache.org/jira/browse/BEAM-10137),
[BEAM-10138](https://issues.apache.org/jira/browse/BEAM-10138)
-* Java PubSubIO - In progress -
[BEAM-7738](https://issues.apache.org/jira/browse/BEAM-7738)
-* Java SnowflakeIO - completed -
[BEAM-9897](https://issues.apache.org/jira/browse/BEAM-9897),
[BEAM-9898](https://issues.apache.org/jira/browse/BEAM-9898)
-* Java SpannerIO - In progress -
[BEAM-10139](https://issues.apache.org/jira/browse/BEAM-10139),
[BEAM-10140](https://issues.apache.org/jira/browse/BEAM-10140)
-* Java SQL - completed -
[BEAM-8603](https://issues.apache.org/jira/browse/BEAM-8603)
-
-### Portable Beam schema
-
-Portable Beam schema support will provide a generalized mechanism for
serializing and transferring data across language boundaries which will be
extremely useful for pipelines that employ cross-language transforms.
-
-* Make row coder a standard coder and implement in python - completed -
[BEAM-7886](https://issues.apache.org/jira/browse/BEAM-7886)
-
-### Integration/Performance testing
-
-* Add an integration test suite for cross-language transforms on Flink runner
- In progress - [BEAM-6683](https://issues.apache.org/jira/browse/BEAM-6683)
-
-### Documentation
-
-Work related to adding documenting on cross-language transforms to Beam
Website.
-
-* Document cross-language transforms API for Java/Python - Not started
-* Document API for making existing transforms available as cross-language
transforms for Java/Python - Not started
+See the full list of available connectors in the [Beam Connectors
Overview](/documentation/io/connectors/).
diff --git a/website/www/site/content/en/roadmap/connectors-python-sdk.md
b/website/www/site/content/en/roadmap/connectors-python-sdk.md
deleted file mode 100644
index 66ca4c956ff..00000000000
--- a/website/www/site/content/en/roadmap/connectors-python-sdk.md
+++ /dev/null
@@ -1,29 +0,0 @@
----
-title: "Connectors - Python SDK"
----
-<!--
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
--->
-
-Roadmap for connectors developed using Python SDK.
-
-# Kafka
-
-An Apache Kafka connectors for Python SDK that is fully developed using
-Splittable DoFn API is planned. This is partially blocked till
-Splittable DoFn work related to portability framework is finalized.
-See [BEAM-3788](https://issues.apache.org/jira/browse/BEAM-3788) for more
details.
-
-# Parquet
-A Python connector for Parquet file format is currently in development.
-See [BEAM-4444](https://issues.apache.org/jira/browse/BEAM-4444) for more
details.
diff --git a/website/www/site/content/en/roadmap/dataflow-runner.md
b/website/www/site/content/en/roadmap/dataflow-runner.md
index 1c096255029..4f259ae11b6 100644
--- a/website/www/site/content/en/roadmap/dataflow-runner.md
+++ b/website/www/site/content/en/roadmap/dataflow-runner.md
@@ -17,7 +17,10 @@ limitations under the License.
# Google Cloud Dataflow Runner Roadmap
-This roadmap is in progress. In the meantime, here are available resources:
+Please check GitHub Issue tracker for recent developments on this topic:
- - [Runner documentation](/documentation/runners/dataflow)
- - Issues:
[runner-dataflow](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Arunner-dataflow)
+ - Issues:
[runner-dataflow](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Adataflow)
+
+Available resources:
+
+- [Runner documentation](/documentation/runners/dataflow)
diff --git a/website/www/site/content/en/roadmap/euphoria.md
b/website/www/site/content/en/roadmap/euphoria.md
index c3c0accff9d..f1ca5866a0a 100644
--- a/website/www/site/content/en/roadmap/euphoria.md
+++ b/website/www/site/content/en/roadmap/euphoria.md
@@ -17,6 +17,8 @@ limitations under the License.
# Euphoria API
+**Note** Euphoria is deprecated and scheduled for removal in a future release.
+
Easy to use Java 8 DSL for the Beam Java SDK. Provides a high-level
abstraction of Beam transformations, which is both easy to read and write. Can
be used as a complement to existing Beam pipelines (convertible back and
forth). You can have a glimpse of the API at [WordCount
example](/documentation/sdks/java/euphoria/#wordcount-example).
- Issues:
[dsl-euphoria](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Adsl-euphoria)
/ [BEAM-3900](https://issues.apache.org/jira/browse/BEAM-3900)
diff --git a/website/www/site/content/en/roadmap/flink-runner.md
b/website/www/site/content/en/roadmap/flink-runner.md
index fbf0f99f807..de0b0015a13 100644
--- a/website/www/site/content/en/roadmap/flink-runner.md
+++ b/website/www/site/content/en/roadmap/flink-runner.md
@@ -17,7 +17,21 @@ limitations under the License.
# Apache Flink Runner Roadmap
-This roadmap is in progress. In the meantime, here are available resources:
+_Last updated on Aug 2026._
- - [Runner documentation](/documentation/runners/flink)
- - Issues:
[runner-flink](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Arunner-flink)
+## Flink versions
+
+Beam aims to support latest Flink minor versions. See [Flink Runner Support
Table](/documentation/runners/flink#flink-version-compatibility)
+for current coverage.
+
+### Flink 2.x Support
+
+One of the major changes in Flink 2 is the removal of Flink DataSet API. As a
result Beam Flink 2 runners switched to use DataStream API for batch mode. One
area of interest is to improve the batch support of DataStream API.
+
+## Available resources
+
+Please check GitHub Issue tracker for recent developments on this topic:
+
+ - Issues:
[runner-flink](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Aflink)
+
+- [Runner documentation](/documentation/runners/flink)
diff --git a/website/www/site/content/en/roadmap/go-sdk.md
b/website/www/site/content/en/roadmap/go-sdk.md
index eb16b7936f7..39bded291eb 100644
--- a/website/www/site/content/en/roadmap/go-sdk.md
+++ b/website/www/site/content/en/roadmap/go-sdk.md
@@ -1,5 +1,7 @@
---
title: "Go SDK Roadmap"
+aliases:
+ - /roadmap/connectors-go-sdk/
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
@@ -17,18 +19,23 @@ limitations under the License.
# Go SDK Roadmap
+_Last updated on Aug 2026._
+
The Go SDK is [fully released as of v2.33.0](/blog/go-sdk-release/).
-The Go SDK the first SDK purely on the [Beam Portability
Framework](/roadmap/portability/)
+The Go SDK is the first SDK purely on the [Beam Portability
Framework](/roadmap/portability/)
and can execute pipelines on portable runners, like Flink, Spark, and Google
Cloud Dataflow.
Current roadmap:
-* continue building up unbounded pipeline facing features, as described on the
[Beam Dev
Wiki](https://cwiki.apache.org/confluence/display/BEAM/Supporting+Streaming+in+the+Go+SDK).
-* improve IO support via cross language transforms, and add scalable native
transforms. [Go SDK Connector Roadmap](/roadmap/connectors-go-sdk/)
+* Continue building up unbounded pipeline facing features, as described on the
[Beam Dev
Wiki](https://cwiki.apache.org/confluence/display/BEAM/Supporting+Streaming+in+the+Go+SDK).
+* Improve I/O support via cross-language transforms:
+ * Complete Beam portable schema support for Go SDK
([Issue](https://github.com/apache/beam/issues/39684)).
+ * Utilize currently available Java and Python connectors through the
cross-language transforms feature.
+ * Support auto-generated cross-language transform wrappers
([doc](https://s.apache.org/autogen-wrappers)), currently supported by Python.
Otherwise, improving examples and documentation for devs and users alike is
ongoing.
Contributions are welcome. Please contact the [dev
list](mailto:[email protected]?subject=%5BGo%20SDK%5D%20How%20can%20I%20help%3F)
for assistance in finding a place to help out.
- - Issues:
[sdk-go](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Asdk-go)
+ - Issues:
[sdk-go](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Ago)
- Contact: Robert Burke (@lostluck)
[Email](mailto:[email protected]?subject=%5BGo%20SDK%20Roadmap%5D) - Please
also cc the [email protected] list. I strongly prefer public discussion of
Go SDK matters.
diff --git a/website/www/site/content/en/roadmap/java-sdk.md
b/website/www/site/content/en/roadmap/java-sdk.md
index d40dabec20f..9b469ec28e3 100644
--- a/website/www/site/content/en/roadmap/java-sdk.md
+++ b/website/www/site/content/en/roadmap/java-sdk.md
@@ -17,9 +17,10 @@ limitations under the License.
# Java SDK Roadmap
-## Next Java LTS version support (Java 25)
+_Last updated on Aug 2026._
-Work to support the next LTS release of Java is in progress. For more details
-about the scope and info on the various tasks please see the GitHub Issue.
+## Next Java LTS version support
-- GitHub: [#35627](https://github.com/apache/beam/issues/35627)
+Beam aims to support new Java LTS versions in a timely manner.
+
+Work to support the next LTS release of Java is done as of Beam 2.69.0. For
more details, see [Java Version Compatibility
matrix](/documentation/sdks/java/#java-version-compatibility).
diff --git a/website/www/site/content/en/roadmap/prism-runner.md
b/website/www/site/content/en/roadmap/prism-runner.md
index 7f741b94d2a..7f4ae9940c8 100644
--- a/website/www/site/content/en/roadmap/prism-runner.md
+++ b/website/www/site/content/en/roadmap/prism-runner.md
@@ -17,6 +17,8 @@ limitations under the License.
# Apache Beam Prism Runner Roadmap
+_Last updated on Aug 2026._
+
The goal for the Prism runner is to provide a good default onboarding
experience for Apache Beam.
* Prism should be able to execute any Beam pipeline that can execute on a
local machine.
diff --git a/website/www/site/content/en/roadmap/python-sdk.md
b/website/www/site/content/en/roadmap/python-sdk.md
index 36a5cf3df4d..d15e97d653f 100644
--- a/website/www/site/content/en/roadmap/python-sdk.md
+++ b/website/www/site/content/en/roadmap/python-sdk.md
@@ -1,5 +1,7 @@
---
title: "Python SDK Roadmap"
+aliases:
+ - /roadmap/connectors-python-sdk/
---
<!--
Licensed under the Apache License, Version 2.0 (the "License");
@@ -17,20 +19,37 @@ limitations under the License.
# Python SDK Roadmap
-## Python 3 Support
+_Last updated on Aug 2026._
-Apache Beam 2.64.0 and higher support Python 3.9, 3.10, 3.11 and 3.12. We're
continuing to
[improve](https://issues.apache.org/jira/browse/BEAM-1251?focusedCommentId=16890504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-1689050)
the experience for Python 3 users and add support for Python 3.x minor
versions:
+## Python Version Support
+Apache Beam 2.73.0 and higher support Python 3.10, 3.11, 3.12, 3.13 and 3.14.
Beam aims to support new Python minor versions in a timely manner:
- - [Kanban
Board](https://issues.apache.org/jira/secure/RapidBoard.jspa?rapidView=245&view=detail)
- - [Python 3 Conversion Quick Start
Guide](https://docs.google.com/document/d/1s1BJVCY65LB_SYK1SU1u7NbZiFANoq-nEYaEvzRbYlA)
- - [Tracking Issue](https://issues.apache.org/jira/browse/BEAM-1251)
- - [Original
Proposal](https://docs.google.com/document/d/1xDG0MWVlDKDPu_IW9gtMvxi2S9I0GB0VDTkPhjXT0nE)
+ - [Python Version Compatibility
matrix](/documentation/sdks/python/#python-version-compatibility)
+
+## Cross-Language Connectors and Auto-Generated Wrappers
+
+Rather than reimplementing storage connectors natively from scratch in Python,
the preferred approach for Beam data connectors is leveraging mature Java I/O
connectors via Beam's cross-language (xlang) framework:
+
+* **Preferred Cross-Language Path**: Core I/O connectors (such as Kafka,
Apache Iceberg, Delta Lake, Snowflake, and JDBC) are authored as
[SchemaTransforms](/documentation/sdks/python-custom-multi-language-pipelines-guide)
in Java and exposed to Python pipelines. Python users can invoke them
seamlessly or configure them via the simplified [Managed
I/O](/documentation/io/managed-io) API.
+* **Auto-Generated Transform Wrappers**: The Python SDK natively supports
generating wrappers for external transforms ([design
doc](https://s.apache.org/autogen-wrappers)). By discovering SchemaTransforms
registered in expansion services, the Python SDK automatically provides
idiomatic, typed Python wrappers and documentation without requiring developers
to manually write and maintain boilerplate code.
+
+## AI and Machine Learning (Beam ML)
+
+Machine learning and artificial intelligence represent the primary focus area
for native Python transform and connector development:
+
+* **[RunInference](/documentation/ml/overview/)**: High-throughput,
production-grade model inference supporting leading ML frameworks (including
PyTorch, TensorFlow, TensorRT, ONNX, and Hugging Face) in both batch and
streaming pipelines.
+* **Vector Database Connectors**: Native integrations and configurations for
vector databases (such as Qdrant, Pinecone, and Vertex AI Vector Search),
supporting large-scale embedding generation and Retrieval-Augmented Generation
(RAG) workflows.
+* **LLM & Multi-Model Workflows**: Native patterns for model evaluation,
multi-model ensemble pipelines, and orchestration with modern AI systems.
+
+Learn more about Beam's machine learning capabilities in the [Beam ML
Documentation](/documentation/ml/overview/).
+
+## Contributions and Feedback
Contributions and feedback are welcome!
-If you are interested in helping, you can select an unassigned issue on the
Kanban board and assign it to yourself. If you cannot assign the issue to
yourself, comment on the issue. When submitting a new PR, please tag
[@aaltay](https://github.com/aaltay), and
[@tvalentyn](https://github.com/tvalentyn).
+If you are interested in helping, you can select an unassigned issue on the
Kanban board and assign it to yourself. If you cannot assign the issue to
yourself, comment on the issue. When submitting a new PR, please tag
[@damccorm](https://github.com/damccorm), and
[@tvalentyn](https://github.com/tvalentyn).
-To report a Python 3 related issue, create a subtask in
[BEAM-1251](https://issues.apache.org/jira/browse/BEAM-1251) and cc: [~altay]
and [~tvalentyn] in a JIRA comment. The best way to help us identify and
investigate the issue is with a minimal pipeline that reproduces the issue.
+To report a Python related issue, create a GitHub Issue in [Python
label](https://github.com/apache/beam/issues?q=is%3Aissue%20state%3Aopen%20label%3Apython)
and cc: [~damccorm] and [~tvalentyn] in a comment. The best way to help us
identify and investigate the issue is with a minimal pipeline that reproduces
the issue.
You can also discuss encountered issues on user@ or dev@ mailing lists as
appropriate.
diff --git a/website/www/site/content/en/roadmap/spark-runner.md
b/website/www/site/content/en/roadmap/spark-runner.md
index 06f4dce8aca..8a7b3e57c95 100644
--- a/website/www/site/content/en/roadmap/spark-runner.md
+++ b/website/www/site/content/en/roadmap/spark-runner.md
@@ -17,10 +17,18 @@ limitations under the License.
# Apache Spark Runner Roadmap
-## Spark 3
+_Last updated on Aug 2026._
-Support for Spark 3 in Beam's Spark runner is ongoing. For info on the various
-tasks please refer to the JIRA ticket.
+## Spark 4
-- JIRA: [BEAM-7093](https://issues.apache.org/jira/browse/BEAM-7093)
-- Contact: [Ismaël Mejía](mailto:[email protected])
+Support for Spark 4 in Beam's Spark runner is ongoing.
+
+As of Beam 2.74.0, Spark 4 is supported by Classic Spark runner in batch mode.
+
+As of Spark 4, DStream API is deprecated and community interest is around
Structured Streaming. We are working on improving the Structured Streaming
support in Beam's Spark runner, this includes
+- Complete streaming support for Spark 4 based on Structured Streaming
+- Build Spark 4 portable runner based on Structured Streaming
+
+For info on the various tasks please refer to the GitHub Issue.
+
+- Issues:
[runner-spark](https://github.com/apache/beam/issues?q=is%3Aopen+is%3Aissue+label%3Aspark)
diff --git a/website/www/site/layouts/partials/section-menu/en/roadmap.html
b/website/www/site/layouts/partials/section-menu/en/roadmap.html
index 09c326705d9..bbb43ba34a8 100644
--- a/website/www/site/layouts/partials/section-menu/en/roadmap.html
+++ b/website/www/site/layouts/partials/section-menu/en/roadmap.html
@@ -14,13 +14,13 @@
<li><a href="/roadmap/">Roadmap Highlights</a></li>
<li><a href="/roadmap/portability">Portability Framework</a></li>
<li class="section-nav-item">
- <span class="section-nav-list-title">Languages</span>
+ <span class="section-nav-list-title">SDKs and Languages</span>
<ul class="section-nav-list">
<li><a href="/roadmap/java-sdk">Java SDK</a></li>
<li><a href="/roadmap/go-sdk">Go SDK</a></li>
<li><a href="/roadmap/python-sdk">Python SDK</a></li>
- <li><a href="/roadmap/euphoria">Euphoria API</a></li>
+ <li><a href="/roadmap/connectors-multi-sdk/">Multi-SDK Efforts</a></li>
<li><a href="/roadmap/sql">SQL</a></li>
</ul>
</li>
@@ -31,18 +31,14 @@
<li><a href="/roadmap/dataflow-runner/">Google Cloud Dataflow
Runner</a></li>
<li><a href="/roadmap/prism-runner/">Prism Runner</a></li>
<li><a href="/roadmap/flink-runner/">Flink Runner</a></li>
- <li><a href="/roadmap/nemo-runner/">Nemo Runner</a></li>
<li><a href="/roadmap/spark-runner/">Spark Runner</a></li>
- <li><a href="/roadmap/twister2-runner/">Twister2 Runner</a></li>
</ul>
</li>
<li class="section-nav-item">
- <span class="section-nav-list-title">Connectors</span>
-
+ <span class="section-nav-list-title">Archived Roadmaps</span>
<ul class="section-nav-list">
- <li><a href="/roadmap/connectors-java-sdk/">Java SDK</a></li>
- <li><a href="/roadmap/connectors-python-sdk/">Python SDK</a></li>
- <li><a href="/roadmap/connectors-go-sdk/">Go SDK</a></li>
- <li><a href="/roadmap/connectors-multi-sdk/">Multi-SDK Efforts</a></li>
+ <li><a href="/roadmap/euphoria">Euphoria API</a></li>
+ <li><a href="/roadmap/nemo-runner/">Nemo Runner</a></li>
+ <li><a href="/roadmap/twister2-runner/">Twister2 Runner</a></li>
</ul>
</li>
