This is an automated email from the ASF dual-hosted git repository.
danoliveira pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 5a351bb5e93 Update Beam website to release 2.38.0 (#17378)
5a351bb5e93 is described below
commit 5a351bb5e93661f65fd519751894be4e4e55d81d
Author: Daniel Oliveira <[email protected]>
AuthorDate: Wed Apr 20 15:48:18 2022 -0700
Update Beam website to release 2.38.0 (#17378)
Update Beam website to release 2.38.0
---
CHANGES.md | 15 +-
website/www/site/config.toml | 2 +-
website/www/site/content/en/blog/beam-2.38.0.md | 164 +++++++++++++++++++++
.../www/site/content/en/get-started/downloads.md | 50 ++++---
4 files changed, 195 insertions(+), 36 deletions(-)
diff --git a/CHANGES.md b/CHANGES.md
index 5dd1fde25a4..8168402c859 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -82,19 +82,11 @@
* ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
-# [2.38.0] - Unreleased
-
-## Highlights
-
-* New highly anticipated feature X added to Python SDK
([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
-* New highly anticipated feature Y added to Java SDK
([BEAM-Y](https://issues.apache.org/jira/browse/BEAM-Y)).
+# [2.38.0] - 2022-04-20
## I/Os
* Introduce projection pushdown optimizer to the Java SDK
([BEAM-12976](https://issues.apache.org/jira/browse/BEAM-12976)). The optimizer
currently only works on the [BigQuery Storage
API](https://beam.apache.org/documentation/io/built-in/google-bigquery/#storage-api),
but more I/Os will be added in future releases. If you encounter a bug with
the optimizer, please file a JIRA and disable the optimizer using pipeline
option `--experiments=disable_projection_pushdown`.
-
* A new IO for Neo4j graph databases was added.
([BEAM-1857](https://issues.apache.org/jira/browse/BEAM-1857)) It has the
ability to update nodes and relationships using UNWIND statements and to read
data using cypher statements with parameters.
-
-* Support for X source added (Java/Python)
([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
* `amazon-web-services2` has reached feature parity and is finally recommended
over the earlier `amazon-web-services` and `kinesis` modules (Java). These will
be deprecated in one of the next releases
([BEAM-13174](https://issues.apache.org/jira/browse/BEAM-13174)).
* Long outstanding write support for `Kinesis` was added
([BEAM-13175](https://issues.apache.org/jira/browse/BEAM-13175)).
* Configuration was simplified and made consistent across all IOs, including
the usage of `AwsOptions`
([BEAM-13563](https://issues.apache.org/jira/browse/BEAM-13563),
[BEAM-13663](https://issues.apache.org/jira/browse/BEAM-13663),
[BEAM-13587](https://issues.apache.org/jira/browse/BEAM-13587)).
@@ -119,7 +111,6 @@
* Beam 2.38.0 will be the last minor release to support Flink 1.11.
* (`amazon-web-services2`) Client providers (`withXYZClientProvider()`) as
well as IO specific `RetryConfiguration`s are deprecated, instead use
`withClientConfiguration()` or `AwsOptions` to configure AWS IOs / clients.
Custom implementations of client providers shall be replaced with a
respective `ClientBuilderFactory` and configured through `AwsOptions`
([BEAM-13563](https://issues.apache.org/jira/browse/BEAM-13563)).
-* X behavior is deprecated and will be removed in X versions
([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
## Bugfixes
@@ -128,10 +119,6 @@
* This caused unnecessarily long pre-processing times before job submission
for large complex pipelines.
* Fix `pyarrow` version parsing
(Python)([BEAM-14235](https://issues.apache.org/jira/browse/BEAM-14235))
-## Known Issues
-
-* ([BEAM-X](https://issues.apache.org/jira/browse/BEAM-X)).
-
# [2.37.0] - 2022-03-04
## Highlights
diff --git a/website/www/site/config.toml b/website/www/site/config.toml
index e9d927ea84d..b4f036ee171 100644
--- a/website/www/site/config.toml
+++ b/website/www/site/config.toml
@@ -104,7 +104,7 @@ github_project_repo = "https://github.com/apache/beam";
[params]
description = "Apache Beam is an open source, unified model and set of
language-specific SDKs for defining and executing data processing workflows,
and also data ingestion and integration flows, supporting Enterprise
Integration Patterns (EIPs) and Domain Specific Languages (DSLs). Dataflow
pipelines simplify the mechanics of large-scale batch and streaming data
processing and can run on a number of runtimes like Apache Flink, Apache Spark,
and Google Cloud Dataflow (a cloud service). Be [...]
-release_latest = "2.37.0"
+release_latest = "2.38.0"
# The repository and branch where the files live in Github or Colab. This is
used
# to serve and stage from your local branch, but publish to the master branch.
# e.g. https://github.com/{{< param branch_repo >}}/path/to/notebook.ipynb
diff --git a/website/www/site/content/en/blog/beam-2.38.0.md
b/website/www/site/content/en/blog/beam-2.38.0.md
new file mode 100644
index 00000000000..f0f12cce733
--- /dev/null
+++ b/website/www/site/content/en/blog/beam-2.38.0.md
@@ -0,0 +1,164 @@
+---
+title: "Apache Beam 2.38.0"
+date: 2022-04-20 9:00:00 -0700
+categories:
+ - blog
+authors:
+ - danoliveira
+---
+
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+We are happy to present the new 2.38.0 release of Beam.
+This release includes both improvements and new functionality.
+See the [download page](/get-started/downloads/#2380-2022-04-20) for this
release.
+
+<!--more-->
+
+For more information on changes in 2.38.0 check out the [detailed release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12351169).
+
+## I/Os
+* Introduce projection pushdown optimizer to the Java SDK
([BEAM-12976](https://issues.apache.org/jira/browse/BEAM-12976)). The optimizer
currently only works on the [BigQuery Storage
API](https://beam.apache.org/documentation/io/built-in/google-bigquery/#storage-api),
but more I/Os will be added in future releases. If you encounter a bug with
the optimizer, please file a JIRA and disable the optimizer using pipeline
option `--experiments=disable_projection_pushdown`.
+* A new IO for Neo4j graph databases was added.
([BEAM-1857](https://issues.apache.org/jira/browse/BEAM-1857)) It has the
ability to update nodes and relationships using UNWIND statements and to read
data using cypher statements with parameters.
+* `amazon-web-services2` has reached feature parity and is finally recommended
over the earlier `amazon-web-services` and `kinesis` modules (Java). These will
be deprecated in one of the next releases
([BEAM-13174](https://issues.apache.org/jira/browse/BEAM-13174)).
+ * Long outstanding write support for `Kinesis` was added
([BEAM-13175](https://issues.apache.org/jira/browse/BEAM-13175)).
+ * Configuration was simplified and made consistent across all IOs, including
the usage of `AwsOptions`
([BEAM-13563](https://issues.apache.org/jira/browse/BEAM-13563),
[BEAM-13663](https://issues.apache.org/jira/browse/BEAM-13663),
[BEAM-13587](https://issues.apache.org/jira/browse/BEAM-13587)).
+ * Additionally, there's a long list of recent improvements and fixes to
+ `S3` Filesystem
([BEAM-13245](https://issues.apache.org/jira/browse/BEAM-13245),
[BEAM-13246](https://issues.apache.org/jira/browse/BEAM-13246),
[BEAM-13441](https://issues.apache.org/jira/browse/BEAM-13441),
[BEAM-13445](https://issues.apache.org/jira/browse/BEAM-13445),
[BEAM-14011](https://issues.apache.org/jira/browse/BEAM-14011)),
+ `DynamoDB` IO
([BEAM-13209](https://issues.apache.org/jira/browse/BEAM-13009),
[BEAM-13209](https://issues.apache.org/jira/browse/BEAM-13209)),
+ `SQS` IO ([BEAM-13631](https://issues.apache.org/jira/browse/BEAM-13631),
[BEAM-13510](https://issues.apache.org/jira/browse/BEAM-13510)) and others.
+
+## New Features / Improvements
+
+* Pipeline dependencies supplied through `--requirements_file` will now be
staged to the runner using binary distributions (wheels) of the PyPI packages
for linux_x86_64 platform
([BEAM-4032](https://issues.apache.org/jira/browse/BEAM-4032)). To restore the
behavior to use source distributions, set pipeline option
`--requirements_cache_only_sources`. To skip staging the packages at submission
time, set pipeline option `--requirements_cache=skip` (Python).
+* The Flink runner now supports Flink 1.14.x
([BEAM-13106](https://issues.apache.org/jira/browse/BEAM-13106)).
+* Interactive Beam now supports remotely executing Flink pipelines on Dataproc
(Python) ([BEAM-14071](https://issues.apache.org/jira/browse/BEAM-14071)).
+
+## Breaking Changes
+
+* (Python) Previously `DoFn.infer_output_types` was expected to return
`Iterable[element_type]` where `element_type` is the PCollection elemnt type.
It is now expected to return `element_type`. Take care if you have overriden
`infer_output_type` in a `DoFn` (this is not common). See
[BEAM-13860](https://issues.apache.org/jira/browse/BEAM-13860).
+* (`amazon-web-services2`) The types of `awsRegion` / `endpoint` in
`AwsOptions` changed from String to `Region` / `URI`
([BEAM-13563](https://issues.apache.org/jira/browse/BEAM-13563)).
+
+## Deprecations
+
+* Beam 2.38.0 will be the last minor release to support Flink 1.11.
+* (`amazon-web-services2`) Client providers (`withXYZClientProvider()`) as
well as IO specific `RetryConfiguration`s are deprecated, instead use
`withClientConfiguration()` or `AwsOptions` to configure AWS IOs / clients.
+ Custom implementations of client providers shall be replaced with a
respective `ClientBuilderFactory` and configured through `AwsOptions`
([BEAM-13563](https://issues.apache.org/jira/browse/BEAM-13563)).
+
+## Bugfixes
+
+* Fix S3 copy for large objects (Java)
([BEAM-14011](https://issues.apache.org/jira/browse/BEAM-14011))
+* Fix quadratic behavior of pipeline canonicalization (Go)
([BEAM-14128](https://issues.apache.org/jira/browse/BEAM-14128))
+ * This caused unnecessarily long pre-processing times before job submission
for large complex pipelines.
+* Fix `pyarrow` version parsing
(Python)([BEAM-14235](https://issues.apache.org/jira/browse/BEAM-14235))
+
+### Known Issues
+
+* See a full list of open [issues that
affect](https://issues.apache.org/jira/issues/?jql=project%20%3D%20BEAM%20AND%20affectedVersion%20%3D%202.38.0%20ORDER%20BY%20priority%20DESC%2C%20updated%20DESC)
this version.
+
+## List of Contributors
+
+According to git shortlog, the following people contributed to the 2.38.0
release. Thank you to all contributors!
+
+abhijeet-lele
+Ahmet Altay
+akustov
+Alexander
+Alexander Zhuravlev
+Alexey Romanenko
+AlikRodriguez
+Anand Inguva
+andoni-guzman
+andreukus
+Andy Ye
+Ankur Goenka
+ansh0l
+Artur Khanin
+Aydar Farrakhov
+Aydar Zainutdinov
+Benjamin Gonzalez
+Brian Hulette
+brucearctor
+bulat safiullin
+bullet03
+Carl Mastrangelo
+Chamikara Jayalath
+Chun Yang
+Daniela Martín
+Daniel Oliveira
+Danny McCormick
+daria.malkova
+David Cavazos
+David Huntsperger
+dmitryor
+Dmytro Sadovnychyi
+dpcollins-google
+egalpin
+Elias Segundo Antonio
+emily
+Etienne Chauchot
+Hengfeng Li
+Ismaël Mejía
+Israel Herraiz
+Jack McCluskey
+Jakub Kukul
+Janek Bevendorff
+Jeff Klukas
+Johan Sternby
+Kamil Breguła
+Kenneth Knowles
+Ke Wu
+Kiley
+Kyle Weaver
+laraschmidt
+Lara Schmidt
+LE QUELLEC Olivier
+Luka Kalinovcic
+Luke Cwik
+Marcin Kuthan
+masahitojp
+Masato Nakamura
+Matt Casters
+Melissa Pashniak
+Michael Li
+Miguel Hernandez
+Moritz Mack
+mosche
+nancyxu123
+Nathan J Mehl
+Niel Markwick
+Ning Kang
+Pablo Estrada
+paul-tlh
+Pavel Avilov
+Rahul Iyer
+Reuven Lax
+Ritesh Ghorse
+Robert Bradshaw
+Robert Burke
+Ryan Skraba
+Ryan Thompson
+Sam Whittle
+Seth Vargo
+sp029619
+Steven Niemitz
+Thiago Nunes
+Udi Meiri
+Valentyn Tymofieiev
+Victor
+vitaly.terentyev
+Yichi Zhang
+Yi Hu
+yirutang
+Zachary Houfek
+Zoe
diff --git a/website/www/site/content/en/get-started/downloads.md
b/website/www/site/content/en/get-started/downloads.md
index 895efb80165..a3a0e27ea62 100644
--- a/website/www/site/content/en/get-started/downloads.md
+++ b/website/www/site/content/en/get-started/downloads.md
@@ -88,58 +88,66 @@ versions denoted `0.x.y`.
## Releases
+### 2.38.0 (2022-04-20)
+Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.38.0/apache-beam-2.38.0-source-release.zip).
+[SHA-512](https://downloads.apache.org/beam/2.38.0/apache-beam-2.38.0-source-release.zip.sha512).
+[signature](https://downloads.apache.org/beam/2.38.0/apache-beam-2.38.0-source-release.zip.asc).
+
+[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12351169)
+[Blog post](/blog/beam-2.38.0).
+
### 2.37.0 (2022-03-04)
-Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.37.0/apache-beam-2.37.0-source-release.zip).
-[SHA-512](https://downloads.apache.org/beam/2.37.0/apache-beam-2.37.0-source-release.zip.sha512).
-[signature](https://downloads.apache.org/beam/2.37.0/apache-beam-2.37.0-source-release.zip.asc).
+Official [source code
download](https://archive.apache.org/dist/beam/2.37.0/apache-beam-2.37.0-source-release.zip).
+[SHA-512](https://archive.apache.org/dist/beam/2.37.0/apache-beam-2.37.0-source-release.zip.sha512).
+[signature](https://archive.apache.org/dist/beam/2.37.0/apache-beam-2.37.0-source-release.zip.asc).
[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12351168)
[Blog post](/blog/beam-2.37.0).
### 2.36.0 (2022-02-07)
-Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.36.0/apache-beam-2.36.0-source-release.zip).
-[SHA-512](https://downloads.apache.org/beam/2.36.0/apache-beam-2.36.0-source-release.zip.sha512).
-[signature](https://downloads.apache.org/beam/2.36.0/apache-beam-2.36.0-source-release.zip.asc).
+Official [source code
download](https://archive.apache.org/dist/beam/2.36.0/apache-beam-2.36.0-source-release.zip).
+[SHA-512](https://archive.apache.org/dist/beam/2.36.0/apache-beam-2.36.0-source-release.zip.sha512).
+[signature](https://archive.apache.org/dist/beam/2.36.0/apache-beam-2.36.0-source-release.zip.asc).
[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350407)
[Blog post](/blog/beam-2.36.0).
### 2.35.0 (2021-12-29)
-Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.35.0/apache-beam-2.35.0-source-release.zip).
-[SHA-512](https://downloads.apache.org/beam/2.35.0/apache-beam-2.35.0-source-release.zip.sha512).
-[signature](https://downloads.apache.org/beam/2.35.0/apache-beam-2.35.0-source-release.zip.asc).
+Official [source code
download](https://archive.apache.org/dist/beam/2.35.0/apache-beam-2.35.0-source-release.zip).
+[SHA-512](https://archive.apache.org/dist/beam/2.35.0/apache-beam-2.35.0-source-release.zip.sha512).
+[signature](https://archive.apache.org/dist/beam/2.35.0/apache-beam-2.35.0-source-release.zip.asc).
[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350406)
[Blog post](/blog/beam-2.35.0).
### 2.34.0 (2021-11-11)
-Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.34.0/apache-beam-2.34.0-source-release.zip).
-[SHA-512](https://downloads.apache.org/beam/2.34.0/apache-beam-2.34.0-source-release.zip.sha512).
-[signature](https://downloads.apache.org/beam/2.34.0/apache-beam-2.34.0-source-release.zip.asc).
+Official [source code
download](https://archive.apache.org/dist/beam/2.34.0/apache-beam-2.34.0-source-release.zip).
+[SHA-512](https://archive.apache.org/dist/beam/2.34.0/apache-beam-2.34.0-source-release.zip.sha512).
+[signature](https://archive.apache.org/dist/beam/2.34.0/apache-beam-2.34.0-source-release.zip.asc).
[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350405)
[Blog post](/blog/beam-2.34.0).
### 2.33.0 (2021-10-07)
-Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.33.0/apache-beam-2.33.0-source-release.zip).
-[SHA-512](https://downloads.apache.org/beam/2.33.0/apache-beam-2.33.0-source-release.zip.sha512).
-[signature](https://downloads.apache.org/beam/2.33.0/apache-beam-2.33.0-source-release.zip.asc).
+Official [source code
download](https://archive.apache.org/dist/beam/2.33.0/apache-beam-2.33.0-source-release.zip).
+[SHA-512](https://archive.apache.org/dist/beam/2.33.0/apache-beam-2.33.0-source-release.zip.sha512).
+[signature](https://archive.apache.org/dist/beam/2.33.0/apache-beam-2.33.0-source-release.zip.asc).
[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12350404)
[Blog post](/blog/beam-2.33.0).
### 2.32.0 (2021-08-25)
-Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.32.0/apache-beam-2.32.0-source-release.zip).
-[SHA-512](https://downloads.apache.org/beam/2.32.0/apache-beam-2.32.0-source-release.zip.sha512).
-[signature](https://downloads.apache.org/beam/2.32.0/apache-beam-2.32.0-source-release.zip.asc).
+Official [source code
download](https://archive.apache.org/dist/beam/2.32.0/apache-beam-2.32.0-source-release.zip).
+[SHA-512](https://archive.apache.org/dist/beam/2.32.0/apache-beam-2.32.0-source-release.zip.sha512).
+[signature](https://archive.apache.org/dist/beam/2.32.0/apache-beam-2.32.0-source-release.zip.asc).
[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12349992)
[Blog post](/blog/beam-2.32.0).
### 2.31.0 (2021-07-08)
-Official [source code
download](https://www.apache.org/dyn/closer.cgi/beam/2.31.0/apache-beam-2.31.0-source-release.zip).
-[SHA-512](https://downloads.apache.org/beam/2.31.0/apache-beam-2.31.0-source-release.zip.sha512).
-[signature](https://downloads.apache.org/beam/2.31.0/apache-beam-2.31.0-source-release.zip.asc).
+Official [source code
download](https://archive.apache.org/dist/beam/2.31.0/apache-beam-2.31.0-source-release.zip).
+[SHA-512](https://archive.apache.org/dist/beam/2.31.0/apache-beam-2.31.0-source-release.zip.sha512).
+[signature](https://archive.apache.org/dist/beam/2.31.0/apache-beam-2.31.0-source-release.zip.asc).
[Release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12349991)
[Blog post](/blog/beam-2.31.0).
