dependabot[bot] opened a new pull request, #41: URL: https://github.com/apache/beam-starter-python/pull/41
Bumps [apache-beam](https://github.com/apache/beam) from 2.62.0 to 2.63.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/releases";>apache-beam's releases</a>.</em></p> <blockquote> <h2>Beam 2.63.0 release</h2> <p>We are happy to present the new 2.63.0 release of Beam. This release includes both improvements and new functionality. See the <a href="https://github.com/apache/beam/blob/HEAD/get-started/downloads/%7B$DOWNLOAD_ANCHOR%7D";>download page</a> for this release.</p> <!-- raw HTML omitted --> <p>For more information on changes in 2.63.0, check out the <a href="https://github.com/apache/beam/milestone/27";>detailed release notes</a>.</p> <h2>I/Os</h2> <ul> <li>Support gcs-connector 3.x+ in GcsUtil (<a href="https://redirect.github.com/apache/beam/pull/33368";>#33368</a>)</li> <li>Support for X source added (Java/Python) (<a href="https://github.com/apache/beam/issues/X";>#X</a>).</li> <li>Introduced <code>--groupFilesFileLoad</code> pipeline option to mitigate side-input related issues in BigQueryIO batch FILE_LOAD on certain runners (including Dataflow Runner V2) (Java) (<a href="https://redirect.github.com/apache/beam/pull/33587";>#33587</a>).</li> </ul> <h2>New Features / Improvements</h2> <ul> <li>Add BigQuery vector/embedding ingestion and enrichment components to apache_beam.ml.rag (Python) (<a href="https://redirect.github.com/apache/beam/pull/33413";>#33413</a>).</li> <li>Upgraded to protobuf 4 (Java) (<a href="https://redirect.github.com/apache/beam/issues/33192";>#33192</a>).</li> <li>[GCSIO] Added retry logic to each batch method of the GCS IO (Python) (<a href="https://redirect.github.com/apache/beam/pull/33539";>#33539</a>)</li> <li>[GCSIO] Enable recursive deletion for GCSFileSystem Paths (Python) (<a href="https://redirect.github.com/apache/beam/pull/33611";>#33611</a>).</li> <li>External, Process based Worker Pool support added to the Go SDK container. (<a href="https://redirect.github.com/apache/beam/pull/33572";>#33572</a>) <ul> <li>This is used to enable sidecar containers to run SDK workers for some runners.</li> <li>See <a href="https://beam.apache.org/documentation/runtime/sdk-harness-config/";>https://beam.apache.org/documentation/runtime/sdk-harness-config/</a> for details.</li> </ul> </li> <li>Support the Process Environment for execution in the Go SDK. (<a href="https://redirect.github.com/apache/beam/pull/33651";>#33651</a>)</li> <li>Prism <ul> <li>Prism now uses the same single port for both pipeline submission and execution on workers. Requests are differentiated by worker-id. (<a href="https://redirect.github.com/apache/beam/pull/33438";>#33438</a>) <ul> <li>This avoids port starvation and provides clarity on port use when running Prism in non-local environments.</li> </ul> </li> <li>Support for <a href="https://github.com/RequiresTimeSortedInputs";><code>@​RequiresTimeSortedInputs</code></a> added. (<a href="https://redirect.github.com/apache/beam/issues/33513";>#33513</a>)</li> <li>Initial support for AllowedLateness added. (<a href="https://redirect.github.com/apache/beam/pull/33542";>#33542</a>)</li> <li>The Go SDK's inprocess Prism runner (AKA the Go SDK default runner) now supports non-loopback mode environment types. (<a href="https://redirect.github.com/apache/beam/pull/33572";>#33572</a>)</li> <li>Support the Process Environment for execution in Prism (<a href="https://redirect.github.com/apache/beam/pull/33651";>#33651</a>)</li> <li>Support the AnyOf Environment for execution in Prism (<a href="https://redirect.github.com/apache/beam/pull/33705";>#33705</a>) <ul> <li>This improves support for developing Xlang pipelines, when using a compatible cross language service.</li> </ul> </li> </ul> </li> <li>Partitions are now configurable for the DaskRunner in the Python SDK (<a href="https://redirect.github.com/apache/beam/pull/33805";>#33805</a>).</li> <li>[Dataflow Streaming] Enable Windmill GetWork Response Batching by default (<a href="https://redirect.github.com/apache/beam/pull/33847";>#33847</a>). <ul> <li>With this change user workers will request batched GetWork responses from backend and backend will send multiple WorkItems in the same response proto.</li> <li>The feature can be disabled by passing <code>--windmillRequestBatchedGetWorkResponse=false</code></li> </ul> </li> </ul> <h2>Breaking Changes</h2> <ul> <li>AWS V1 I/Os have been removed (Java). As part of this, x-lang Python Kinesis I/O has been updated to consume the V2 IO and it also no longer supports setting producer_properties (<a href="https://redirect.github.com/apache/beam/issues/33430";>#33430</a>).</li> <li>Upgraded to protobuf 4 (Java) (<a href="https://redirect.github.com/apache/beam/issues/33192";>#33192</a>), but forced Debezium IO to use protobuf 3 (<a href="https://redirect.github.com/apache/beam/issues/33541";>#33541</a> because Debezium clients are not protobuf 4 compatible. This may cause conflicts when using clients which are only compatible with protobuf 4.</li> <li>Minimum Go version for Beam Go updated to 1.22.10 (<a href="https://redirect.github.com/apache/beam/pull/33609";>#33609</a>)</li> </ul> <h2>Bugfixes</h2> <ul> <li>Fix data loss issues when reading gzipped files with TextIO (Python) (<a href="https://redirect.github.com/apache/beam/issues/18390";>#18390</a>, <a href="https://redirect.github.com/apache/beam/issues/31040";>#31040</a>).</li> <li>[BigQueryIO] Fixed an issue where Storage Write API sometimes doesn't pick up auto-schema updates (<a href="https://redirect.github.com/apache/beam/pull/33231";>#33231</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/blob/master/CHANGES.md";>apache-beam's changelog</a>.</em></p> <blockquote> <h1>[2.63.0] - 2025-02-18</h1> <h2>I/Os</h2> <ul> <li>Support gcs-connector 3.x+ in GcsUtil (<a href="https://redirect.github.com/apache/beam/pull/33368";>#33368</a>)</li> <li>Support for X source added (Java/Python) (<a href="https://github.com/apache/beam/issues/X";>#X</a>).</li> <li>Introduced <code>--groupFilesFileLoad</code> pipeline option to mitigate side-input related issues in BigQueryIO batch FILE_LOAD on certain runners (including Dataflow Runner V2) (Java) (<a href="https://redirect.github.com/apache/beam/pull/33587";>#33587</a>).</li> </ul> <h2>New Features / Improvements</h2> <ul> <li>Add BigQuery vector/embedding ingestion and enrichment components to apache_beam.ml.rag (Python) (<a href="https://redirect.github.com/apache/beam/pull/33413";>#33413</a>).</li> <li>Upgraded to protobuf 4 (Java) (<a href="https://redirect.github.com/apache/beam/issues/33192";>#33192</a>).</li> <li>[GCSIO] Added retry logic to each batch method of the GCS IO (Python) (<a href="https://redirect.github.com/apache/beam/pull/33539";>#33539</a>)</li> <li>[GCSIO] Enable recursive deletion for GCSFileSystem Paths (Python) (<a href="https://redirect.github.com/apache/beam/pull/33611";>#33611</a>).</li> <li>External, Process based Worker Pool support added to the Go SDK container. (<a href="https://redirect.github.com/apache/beam/pull/33572";>#33572</a>) <ul> <li>This is used to enable sidecar containers to run SDK workers for some runners.</li> <li>See <a href="https://beam.apache.org/documentation/runtime/sdk-harness-config/";>https://beam.apache.org/documentation/runtime/sdk-harness-config/</a> for details.</li> </ul> </li> <li>Support the Process Environment for execution in the Go SDK. (<a href="https://redirect.github.com/apache/beam/pull/33651";>#33651</a>)</li> <li>Prism <ul> <li>Prism now uses the same single port for both pipeline submission and execution on workers. Requests are differentiated by worker-id. (<a href="https://redirect.github.com/apache/beam/pull/33438";>#33438</a>) <ul> <li>This avoids port starvation and provides clarity on port use when running Prism in non-local environments.</li> </ul> </li> <li>Support for <a href="https://github.com/RequiresTimeSortedInputs";><code>@​RequiresTimeSortedInputs</code></a> added. (<a href="https://redirect.github.com/apache/beam/issues/33513";>#33513</a>)</li> <li>Initial support for AllowedLateness added. (<a href="https://redirect.github.com/apache/beam/pull/33542";>#33542</a>)</li> <li>The Go SDK's inprocess Prism runner (AKA the Go SDK default runner) now supports non-loopback mode environment types. (<a href="https://redirect.github.com/apache/beam/pull/33572";>#33572</a>)</li> <li>Support the Process Environment for execution in Prism (<a href="https://redirect.github.com/apache/beam/pull/33651";>#33651</a>)</li> <li>Support the AnyOf Environment for execution in Prism (<a href="https://redirect.github.com/apache/beam/pull/33705";>#33705</a>) <ul> <li>This improves support for developing Xlang pipelines, when using a compatible cross language service.</li> </ul> </li> </ul> </li> <li>Partitions are now configurable for the DaskRunner in the Python SDK (<a href="https://redirect.github.com/apache/beam/pull/33805";>#33805</a>).</li> <li>[Dataflow Streaming] Enable Windmill GetWork Response Batching by default (<a href="https://redirect.github.com/apache/beam/pull/33847";>#33847</a>). <ul> <li>With this change user workers will request batched GetWork responses from backend and backend will send multiple WorkItems in the same response proto.</li> <li>The feature can be disabled by passing <code>--windmillRequestBatchedGetWorkResponse=false</code></li> </ul> </li> </ul> <h2>Breaking Changes</h2> <ul> <li>AWS V1 I/Os have been removed (Java). As part of this, x-lang Python Kinesis I/O has been updated to consume the V2 IO and it also no longer supports setting producer_properties (<a href="https://redirect.github.com/apache/beam/issues/33430";>#33430</a>).</li> <li>Upgraded to protobuf 4 (Java) (<a href="https://redirect.github.com/apache/beam/issues/33192";>#33192</a>), but forced Debezium IO to use protobuf 3 (<a href="https://redirect.github.com/apache/beam/issues/33541";>#33541</a> because Debezium clients are not protobuf 4 compatible. This may cause conflicts when using clients which are only compatible with protobuf 4.</li> <li>Minimum Go version for Beam Go updated to 1.22.10 (<a href="https://redirect.github.com/apache/beam/pull/33609";>#33609</a>)</li> </ul> <h2>Bugfixes</h2> <ul> <li>Fix data loss issues when reading gzipped files with TextIO (Python) (<a href="https://redirect.github.com/apache/beam/issues/18390";>#18390</a>, <a href="https://redirect.github.com/apache/beam/issues/31040";>#31040</a>).</li> <li>[BigQueryIO] Fixed an issue where Storage Write API sometimes doesn't pick up auto-schema updates (<a href="https://redirect.github.com/apache/beam/pull/33231";>#33231</a>)</li> <li>Prism <ul> <li>Fixed an edge case where Bundle Finalization might not become enabled. (<a href="https://redirect.github.com/apache/beam/issues/33493";>#33493</a>).</li> <li>Fixed session window aggregation, which wasn't being performed per-key. (<a href="https://redirect.github.com/apache/beam/issues/33542";>#33542</a>).)</li> </ul> </li> <li>[Dataflow Streaming Appliance] Fixed commits failing with KeyCommitTooLargeException when a key outputs >180MB of results. <a href="https://redirect.github.com/apache/beam/issues/33588";>#33588</a>.</li> <li>Fixed a Dataflow template creation issue that ignores template file creation errors (Java) (<a href="https://redirect.github.com/apache/beam/issues/33636";>#33636</a>)</li> <li>Correctly documented Pane Encodings in the portability protocols (<a href="https://redirect.github.com/apache/beam/issues/33840";>#33840</a>).</li> <li>Fixed the user mailing list address (<a href="https://redirect.github.com/apache/beam/issues/26013";>#26013</a>).</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/beam/commit/af7f812ec8a241004d489408c8fe9fe9497186be";><code>af7f812</code></a> Set version for 2.63.0 RC2</li> <li><a href="https://github.com/apache/beam/commit/a9fcaafd6bc12e6d1d73bd3dba4737ce9ff0a50b";><code>a9fcaaf</code></a> Skip BoundedTrie on Dataflow till service is have BoundedTrie (<a href="https://redirect.github.com/apache/beam/issues/33921";>#33921</a>) (<a href="https://redirect.github.com/apache/beam/issues/33947";>#33947</a>)</li> <li><a href="https://github.com/apache/beam/commit/c5b4db0fcbf0b812aefd17041eb6a8546034526d";><code>c5b4db0</code></a> Update CHANGES.md for lineage (<a href="https://redirect.github.com/apache/beam/issues/33950";>#33950</a>)</li> <li><a href="https://github.com/apache/beam/commit/5e5ea0705229389dfbefe6d269a2e6d303545809";><code>5e5ea07</code></a> null arrays default to empty array (<a href="https://redirect.github.com/apache/beam/issues/33919";>#33919</a>) (<a href="https://redirect.github.com/apache/beam/issues/33945";>#33945</a>)</li> <li><a href="https://github.com/apache/beam/commit/e285617d30432ab3e13c29a152c6ffbe62672bee";><code>e285617</code></a> Revert emitting lineage as boundedtrie from python (<a href="https://redirect.github.com/apache/beam/issues/33935";>#33935</a>)</li> <li><a href="https://github.com/apache/beam/commit/b08afb50126ba50221c6a8170258bf628205e1cf";><code>b08afb5</code></a> Cherry pick Add support for bounded trie metric in legacy worker (<a href="https://redirect.github.com/apache/beam/issues/33890";>#33890</a>)</li> <li><a href="https://github.com/apache/beam/commit/e790cce5d795f1a408acc728a8057a82ad3a21e4";><code>e790cce</code></a> Add AlloyDB vector writer. (<a href="https://redirect.github.com/apache/beam/issues/33841";>#33841</a>) (<a href="https://redirect.github.com/apache/beam/issues/33880";>#33880</a>)</li> <li><a href="https://github.com/apache/beam/commit/f4b099293e7dfb40083609223148695cb87b27ed";><code>f4b0992</code></a> Update build Go version to 1.23.6 (<a href="https://redirect.github.com/apache/beam/issues/33877";>#33877</a>)</li> <li><a href="https://github.com/apache/beam/commit/49698c33f9a3eb1b37064a4f88069c4bbbea0025";><code>49698c3</code></a> Fix release version (<a href="https://redirect.github.com/apache/beam/issues/33871";>#33871</a>)</li> <li><a href="https://github.com/apache/beam/commit/916c99aa9141ecf7bf60cfbb87c0678dfe8c4f38";><code>916c99a</code></a> Set Dataflow container to release version.</li> <li>Additional commits viewable in <a href="https://github.com/apache/beam/compare/v2.62.0...v2.63.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
