This is an automated email from the ASF dual-hosted git repository.
scwhittle pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git
The following commit(s) were added to refs/heads/master by this push:
new c2c640f8c33 Update CHANGES for recent fixes, improvements. (#32568)
c2c640f8c33 is described below
commit c2c640f8c33071d5bb3e854e82c554c03a0bc851
Author: Sam Whittle <[email protected]>
AuthorDate: Fri Sep 27 12:13:38 2024 +0200
Update CHANGES for recent fixes, improvements. (#32568)
---
CHANGES.md | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/CHANGES.md b/CHANGES.md
index c123a8e1a4d..7c377648c11 100644
--- a/CHANGES.md
+++ b/CHANGES.md
@@ -64,6 +64,7 @@
* Dataflow worker can install packages from Google Artifact Registry Python
repositories (Python) ([#32123](https://github.com/apache/beam/issues/32123)).
* Added support for Zstd codec in SerializableAvroCodecFactory (Java)
([#32349](https://github.com/apache/beam/issues/32349))
* Added support for using vLLM in the RunInference transform (Python)
([#32528](https://github.com/apache/beam/issues/32528))
+* Significantly improved performance of Kafka IO reads that enable
[commitOffsetsInFinalize](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/kafka/KafkaIO.Read.html#commitOffsetsInFinalize--)
by removing the data reshuffle from SDF implementation.
([#31682](https://github.com/apache/beam/pull/31682)).
## Breaking Changes
@@ -72,6 +73,7 @@
as strings rather than silently coerced (and possibly truncated) to numeric
values. To retain the old behavior, pass `dtype=True` (or any other value
accepted by `pandas.read_json`).
+* Users of KafkaIO Read transform that enable
[commitOffsetsInFinalize](https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/kafka/KafkaIO.Read.html#commitOffsetsInFinalize--)
might encounter pipeline graph compatibility issues when updating the
pipeline. To mitigate, set the `updateCompatibilityVersion` option to the SDK
version used for the original pipeline, example
`--updateCompatabilityVersion=2.58.1`
## Deprecations
@@ -81,6 +83,8 @@ when running on 3.8.
([#31192](https://github.com/apache/beam/issues/31192))
## Bugfixes
* (Java) Fixed custom delimiter issues in TextIO
([#32249](https://github.com/apache/beam/issues/32249),
[#32251](https://github.com/apache/beam/issues/32251)).
+* (Java, Python, Go) Fixed PeriodicSequence backlog bytes reporting, which was
preventing Dataflow Runner autoscaling from functioning properly
([#32506](https://github.com/apache/beam/issues/32506)).
+* (Java) Fix improper decoding of rows with schemas containing nullable fields
when encoded with a schema with equal encoding positions but modified field
order. ([#32388](https://github.com/apache/beam/issues/32388)).
## Security Fixes
* Fixed (CVE-YYYY-NNNN)[https://www.cve.org/CVERecord?id=CVE-YYYY-NNNN]
(Java/Python/Go) ([#X](https://github.com/apache/beam/issues/X)).
