This is an automated email from the ASF dual-hosted git repository.
boyuanz pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new 8eb62cb Update beam 2.19.0 release blog
new 2ad1f68 Merge pull request #10722 from boyuanzz/blog
8eb62cb is described below
commit 8eb62cbb00541350bffa386e0862d887da70f1fe
Author: Boyuan Zhang <[email protected]>
AuthorDate: Wed Jan 29 19:52:45 2020 -0800
Update beam 2.19.0 release blog
---
website/src/_data/authors.yml | 5 ++
website/src/_posts/2020-02-04-beam-2.19.0.md | 97 ++++++++++++++++++++++++++++
2 files changed, 102 insertions(+)
diff --git a/website/src/_data/authors.yml b/website/src/_data/authors.yml
index 5812439..93ae2be 100644
--- a/website/src/_data/authors.yml
+++ b/website/src/_data/authors.yml
@@ -139,3 +139,8 @@ udim:
name: Udi Meiri
email: [email protected]
twitter: udim
+boyuanzz:
+ name: Boyuan Zhang
+ email: [email protected]
+ twitter:
+
diff --git a/website/src/_posts/2020-02-04-beam-2.19.0.md
b/website/src/_posts/2020-02-04-beam-2.19.0.md
new file mode 100644
index 0000000..31ffb47
--- /dev/null
+++ b/website/src/_posts/2020-02-04-beam-2.19.0.md
@@ -0,0 +1,97 @@
+---
+layout: post
+title: "Apache Beam 2.19.0"
+date: 2020-02-04 00:00:01 -0800
+# Date above corrected but keep the old URL:
+permalink: /blog/2020/02/04/beam-2.19.0.html
+excerpt_separator: <!--more-->
+categories: blog
+authors:
+ - boyuanzz
+
+---
+<!--
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+-->
+
+We are happy to present the new 2.19.0 release of Beam. This release includes
both improvements and new functionality.
+See the [download page]({{ site.baseurl
}}/get-started/downloads/#2190-2020-02-04) for this release.<!--more-->
+For more information on changes in 2.19.0, check out the
+[detailed release
notes](https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12346582).
+
+## Highlights
+ * Multiple improvements made into Python SDK harness:
+ [BEAM-8624](https://issues.apache.org/jira/browse/BEAM-8624),
+ [BEAM-8623](https://issues.apache.org/jira/browse/BEAM-8623),
+ [BEAM-7949](https://issues.apache.org/jira/browse/BEAM-7949),
+ [BEAM-8935](https://issues.apache.org/jira/browse/BEAM-8935),
+ [BEAM-8816](https://issues.apache.org/jira/browse/BEAM-8816)
+
+### I/Os
+* [BEAM-1440](https://issues.apache.org/jira/browse/BEAM-1440) Create a
BigQuery source (that implements iobase.BoundedSource) for Python SDK
+* [BEAM-2572](https://issues.apache.org/jira/browse/BEAM-2572) Implement an S3
filesystem for Python SDK
+* [BEAM-5192](https://issues.apache.org/jira/browse/BEAM-5192) Support
Elasticsearch 7.x
+* [BEAM-8745](https://issues.apache.org/jira/browse/BEAM-8745) More
fine-grained controls for the size of a BigQuery Load job
+* [BEAM-8801](https://issues.apache.org/jira/browse/BEAM-8801)
PubsubMessageToRow should not check useFlatSchema() in processElement
+* [BEAM-8953](https://issues.apache.org/jira/browse/BEAM-8953) Extend
ParquetIO.Read/ReadFiles.Builder to support Avro GenericData model
+* [BEAM-8946](https://issues.apache.org/jira/browse/BEAM-8946) Report
collection size from MongoDBIOIT
+* [BEAM-8978](https://issues.apache.org/jira/browse/BEAM-8978) Report saved
data size from HadoopFormatIOIT
+
+### New Features / Improvements
+* [BEAM-6008](https://issues.apache.org/jira/browse/BEAM-6008) Improve error
reporting in Java/Python PortableRunner
+* [BEAM-8296](https://issues.apache.org/jira/browse/BEAM-8296) Containerize
the Spark job server
+* [BEAM-8746](https://issues.apache.org/jira/browse/BEAM-8746) Allow the local
job service to work from inside docker
+* [BEAM-8837](https://issues.apache.org/jira/browse/BEAM-8837)
PCollectionVisualizationTest: possible bug
+* [BEAM-8139](https://issues.apache.org/jira/browse/BEAM-8139) Execute
portable Spark application jar
+* [BEAM-9019](https://issues.apache.org/jira/browse/BEAM-9019) Improve Spark
Encoders (wrappers of beam coders)
+* [BEAM-9053](https://issues.apache.org/jira/browse/BEAM-9053) Improve error
message when unable to get the correct filesystem for specified path in Python
SDK) Improve error message when unable to get the correct filesystem for
specified path in Python SDK
+* [BEAM-9055](https://issues.apache.org/jira/browse/BEAM-9055) Unify the
config names of Fn Data API across languages
+
+### SQL
+* [BEAM-5690](https://issues.apache.org/jira/browse/BEAM-5690) Issue with
GroupByKey in BeamSql using SparkRunner
+* [BEAM-8993](https://issues.apache.org/jira/browse/BEAM-8993) [SQL] MongoDb
should use predicate push-down
+* [BEAM-8844](https://issues.apache.org/jira/browse/BEAM-8844) [SQL] Create
performance tests for BigQueryTable
+* [BEAM-9023](https://issues.apache.org/jira/browse/BEAM-9023) Upgrade to
ZetaSQL 2019.12.1
+
+### Breaking Changes
+* [BEAM-8989](https://issues.apache.org/jira/browse/BEAM-8989) Backwards
incompatible change in ParDo.getSideInputs (caught by failure when running
Apache Nemo quickstart)
+* [BEAM-8402](https://issues.apache.org/jira/browse/BEAM-8402) Backwards
incompatible change related to how Environments are represented in Python
`DirectRunner`.
+* [BEAM-9218](https://issues.apache.org/jira/browse/BEAM-9218) Template
staging broken on Beam 2.18.0
+
+### Dependency Changes
+* [BEAM-8696](https://issues.apache.org/jira/browse/BEAM-8696) Beam Dependency
Update Request: com.google.protobuf:protobuf-java
+* [BEAM-8701](https://issues.apache.org/jira/browse/BEAM-8701) Beam Dependency
Update Request: commons-io:commons-io
+* [BEAM-8716](https://issues.apache.org/jira/browse/BEAM-8716) Beam Dependency
Update Request: org.apache.commons:commons-csv
+* [BEAM-8717](https://issues.apache.org/jira/browse/BEAM-8717) Beam Dependency
Update Request: org.apache.commons:commons-lang3
+* [BEAM-8749](https://issues.apache.org/jira/browse/BEAM-8749) Beam Dependency
Update Request: com.datastax.cassandra:cassandra-driver-mapping
+* [BEAM-5546](https://issues.apache.org/jira/browse/BEAM-5546) Beam Dependency
Update Request: commons-codec:commons-codec
+
+### Bugfixes
+* [BEAM-9123](https://issues.apache.org/jira/browse/BEAM-9123)
HadoopResourceId returns wrong directory name
+* [BEAM-8962](https://issues.apache.org/jira/browse/BEAM-8962)
FlinkMetricContainer causes churn in the JobManager and lets the web frontend
malfunction
+* [BEAM-5495](https://issues.apache.org/jira/browse/BEAM-5495)
PipelineResources algorithm is not working in most environments
+* [BEAM-8025](https://issues.apache.org/jira/browse/BEAM-8025) Cassandra IO
classMethod test is flaky
+* [BEAM-8577](https://issues.apache.org/jira/browse/BEAM-8577) FileSystems may
have not be initialized during ResourceId deserialization
+* [BEAM-8582](https://issues.apache.org/jira/browse/BEAM-8582) Python SDK
emits duplicate records for Default and AfterWatermark triggers
+* [BEAM-8943](https://issues.apache.org/jira/browse/BEAM-8943) SDK harness
servers don't shut down properly when SDK harness environment cleanup fails
+* [BEAM-8995](https://issues.apache.org/jira/browse/BEAM-8995)
apache_beam.io.gcp.bigquery_read_it_test failing on Py3.5 PC with: TypeError:
the JSON object must be str, not 'bytes'
+* [BEAM-8999](https://issues.apache.org/jira/browse/BEAM-8999) PGBKCVOperation
does not respect timestamp combiners
+* [BEAM-9050](https://issues.apache.org/jira/browse/BEAM-9050) Beam pickler
doesn't pickle classes that have __module__ set to None.
+*
+* Various bug fixes and performance improvements.
+
+## List of Contributors
+
+According to git shortlog, the following people contributed to the 2.19.0
release. Thank you to all contributors!
+
+Ahmet Altay, Alex Amato, Alexey Romanenko, Andrew Pilloud, Ankur Goenka, Anton
Kedin, Boyuan Zhang, Brian Hulette, Brian Martin, Chamikara Jayalath, Charles
Chen, Craig Chambers, Daniel Oliveira, David Moravek, David Rieber, Dustin
Rhodes, Etienne Chauchot, Gleb Kanterov, Hai Lu, Heejong Lee, Ismaël Mejía, Jan
Lukavský, Jason Kuster, Jean-Baptiste Onofré, Jeff Klukas, João Cabrita, J Ross
Thomson, Juan Rael, Juta, Kasia Kucharczyk, Kengo Seki, Kenneth Jung, Kenneth
Knowles, Kyle Weaver, [...]
