dependabot[bot] opened a new pull request, #40: URL: https://github.com/apache/beam-starter-python/pull/40
Bumps [apache-beam](https://github.com/apache/beam) from 2.61.0 to 2.62.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/releases";>apache-beam's releases</a>.</em></p> <blockquote> <h2>Beam 2.62.0 release</h2> <p>We are happy to present the new 2.62.0 release of Beam. This release includes both improvements and new functionality. See the <a href="https://github.com/apache/beam/blob/HEAD/get-started/downloads/%7B$DOWNLOAD_ANCHOR%7D";>download page</a> for this release.</p> <p>For more information on changes in 2.62.0, check out the <a href="https://github.com/apache/beam/milestone/26";>detailed release notes</a>.</p> <h2>New Features / Improvements</h2> <ul> <li>Added support for stateful processing in Spark Runner for streaming pipelines. Timer functionality is not yet supported and will be implemented in a future release (<a href="https://redirect.github.com/apache/beam/issues/33237";>#33237</a>).</li> <li>The datetime module is now available for use in jinja templatization for yaml.</li> <li>Improved batch performance of SparkRunner's GroupByKey (<a href="https://redirect.github.com/apache/beam/pull/20943";>#20943</a>).</li> <li>Support OnWindowExpiration in Prism (<a href="https://redirect.github.com/apache/beam/issues/32211";>#32211</a>). <ul> <li>This enables initial Java GroupIntoBatches support.</li> </ul> </li> <li>Support OrderedListState in Prism (<a href="https://redirect.github.com/apache/beam/issues/32929";>#32929</a>).</li> </ul> <h2>I/Os</h2> <ul> <li>gcs-connector config options can be set via GcsOptions (Java) (<a href="https://redirect.github.com/apache/beam/pull/32769";>#32769</a>).</li> <li>[Managed Iceberg] Support partitioning by time (year, month, day, hour) for types <code>date</code>, <code>time</code>, <code>timestamp</code>, and <code>timestamp(tz)</code> (<a href="https://redirect.github.com/apache/beam/pull/32939";>#32939</a>)</li> <li>Upgraded the default version of Hadoop dependencies to 3.4.1. Hadoop 2.10.2 is still supported (Java) (<a href="https://redirect.github.com/apache/beam/issues/33011";>#33011</a>).</li> <li>[BigQueryIO] Create managed BigLake tables dynamically (<a href="https://redirect.github.com/apache/beam/pull/33125";>#33125</a>)</li> </ul> <h2>Breaking Changes</h2> <ul> <li>Upgraded ZetaSQL to 2024.11.1 (<a href="https://redirect.github.com/apache/beam/pull/32902";>#32902</a>). Java11+ is now needed if Beam's ZetaSQL component is used.</li> </ul> <h2>Bugfixes</h2> <ul> <li>Fixed EventTimeTimer ordering in Prism. (<a href="https://redirect.github.com/apache/beam/issues/32222";>#32222</a>).</li> <li>[Managed Iceberg] Fixed a bug where DataFile metadata was assigned incorrect partition values (<a href="https://redirect.github.com/apache/beam/pull/33549";>#33549</a>).</li> </ul> <h2>Security Fixes</h2> <ul> <li>Fixed (CVE-2024-47561)[https://www.cve.org/CVERecord?id=CVE-2024-47561] (Java) by upgrading Avro version to 1.11.4</li> </ul> <p>For the most up to date list of known issues, see <a href="https://github.com/apache/beam/blob/master/CHANGES.md";>https://github.com/apache/beam/blob/master/CHANGES.md</a></p> <h2>List of Contributors</h2> <p>According to git shortlog, the following people contributed to the 2.62.0 release. Thank you to all contributors!</p> <p>Ahmed Abualsaud, Ahmet Altay, Alex Merose, Andrew Crites, Arnout Engelen, Attila Doroszlai, Bartosz Zablocki, Chamikara Jayalath, Claire McGinty, Claude van der Merwe, Damon Douglas, Danny McCormick, Gabija Balvociute, Hai Joey Tran, Hakampreet Singh Pandher, Ian Sullivan, Jack McCluskey, Jan Lukavský, Jeff Kinard, Jeffrey Kinard, Laura Detmer, Kenneth Knowles, Martin Trieu, Mattie Fu, Michel Davit, Naireen Hussain, Nick Anikin, Radosław Stankiewicz, Ravi Magham, Reeba Qureshi, Robert Bradshaw, Robert Burke, Rohit Sinha, S. Veyrié, Sam Whittle, Shingo Furuyama, Shunping Huang, Svetak Sundhar, Valentyn Tymofieiev, Vlado Djerek, XQ Hu, Yi Hu, twosom</p> </blockquote> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/apache/beam/blob/master/CHANGES.md";>apache-beam's changelog</a>.</em></p> <blockquote> <h1>[2.62.0] - 2025-01-21</h1> <h2>I/Os</h2> <ul> <li>gcs-connector config options can be set via GcsOptions (Java) (<a href="https://redirect.github.com/apache/beam/pull/32769";>#32769</a>).</li> <li>[Managed Iceberg] Support partitioning by time (year, month, day, hour) for types <code>date</code>, <code>time</code>, <code>timestamp</code>, and <code>timestamp(tz)</code> (<a href="https://redirect.github.com/apache/beam/pull/32939";>#32939</a>)</li> <li>Upgraded the default version of Hadoop dependencies to 3.4.1. Hadoop 2.10.2 is still supported (Java) (<a href="https://redirect.github.com/apache/beam/issues/33011";>#33011</a>).</li> <li>[BigQueryIO] Create managed BigLake tables dynamically (<a href="https://redirect.github.com/apache/beam/pull/33125";>#33125</a>)</li> </ul> <h2>New Features / Improvements</h2> <ul> <li>Added support for stateful processing in Spark Runner for streaming pipelines. Timer functionality is not yet supported and will be implemented in a future release (<a href="https://redirect.github.com/apache/beam/issues/33237";>#33237</a>).</li> <li>The datetime module is now available for use in jinja templatization for yaml.</li> <li>Improved batch performance of SparkRunner's GroupByKey (<a href="https://redirect.github.com/apache/beam/pull/20943";>#20943</a>).</li> <li>Support OnWindowExpiration in Prism (<a href="https://redirect.github.com/apache/beam/issues/32211";>#32211</a>). <ul> <li>This enables initial Java GroupIntoBatches support.</li> </ul> </li> <li>Support OrderedListState in Prism (<a href="https://redirect.github.com/apache/beam/issues/32929";>#32929</a>).</li> <li>Add apache_beam.ml.rag package with RAG types, base chunking, LangChain chunking and HuggingFace embedding components (Python) (<a href="https://redirect.github.com/apache/beam/pull/33364";>#33364</a>).</li> </ul> <h2>Breaking Changes</h2> <ul> <li>Upgraded ZetaSQL to 2024.11.1 (<a href="https://redirect.github.com/apache/beam/pull/32902";>#32902</a>). Java11+ is now needed if Beam's ZetaSQL component is used.</li> </ul> <h2>Bugfixes</h2> <ul> <li>Fixed EventTimeTimer ordering in Prism. (<a href="https://redirect.github.com/apache/beam/issues/32222";>#32222</a>).</li> <li>[Managed Iceberg] Fixed a bug where DataFile metadata was assigned incorrect partition values (<a href="https://redirect.github.com/apache/beam/pull/33549";>#33549</a>).</li> </ul> <h2>Security Fixes</h2> <ul> <li>Fixed (CVE-2024-47561)[https://www.cve.org/CVERecord?id=CVE-2024-47561] (Java) by upgrading Avro version to 1.11.4</li> </ul> <h2>Known Issues</h2> <ul> <li>[Python] If you are using the official Apache Beam Python containers for version 2.62.0, be aware that they include NumPy version 1.26.4. It is strongly recommended that you explicitly specify numpy==1.26.4 in your project's dependency list. (<a href="https://redirect.github.com/apache/beam/issues/33639";>#33639</a>).</li> <li>[Dataflow Streaming Appliance] Commits fail with KeyCommitTooLargeException when a key outputs >180MB of results. Bug affects versions 2.60.0 to 2.62.0, <ul> <li>fix will be released with 2.63.0. <a href="https://redirect.github.com/apache/beam/issues/33588";>#33588</a>.</li> <li>To resolve this issue, downgrade to 2.59.0 or upgrade to 2.63.0 or enable <a href="https://cloud.google.com/dataflow/docs/streaming-engine#use";>Streaming Engine</a>.</li> </ul> </li> </ul> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/apache/beam/commit/df0ad590c16ed993ab97b5641822c1b89564b6b8";><code>df0ad59</code></a> Set version for 2.62.0 RC1</li> <li><a href="https://github.com/apache/beam/commit/77f38cd74735cc732c99d0752ee10c3585a3b0c7";><code>77f38cd</code></a> Cherrypick <a href="https://redirect.github.com/apache/beam/issues/33549";>#33549</a> to Release 2.62.0 branch (Iceberg partition value race cond...</li> <li><a href="https://github.com/apache/beam/commit/0c54e6aea559612f2d6ea9f234f6f4fcb98f1f37";><code>0c54e6a</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/33546";>#33546</a>: Cherry-pick <a href="https://redirect.github.com/apache/beam/issues/33505";>#33505</a> to 2.62.0 release branch (Build...</li> <li><a href="https://github.com/apache/beam/commit/1cc4b4cf8da0c073c52b532293698194bba51001";><code>1cc4b4c</code></a> Build wheels on gha runners instead of local runners (<a href="https://redirect.github.com/apache/beam/issues/33505";>#33505</a>)</li> <li><a href="https://github.com/apache/beam/commit/48e18c48f2b7e71eb477327243ddb2d8e3f8b3fa";><code>48e18c4</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/33535";>#33535</a>: Cherry pick <a href="https://redirect.github.com/apache/beam/issues/33231";>#33231</a> to 2.62.0 release branch ([BigQ...</li> <li><a href="https://github.com/apache/beam/commit/eab83035800f418eb2b9d28512fb3a7c04e2a16e";><code>eab8303</code></a> [BigQueryIO] fetch updated schema for newly created Storage API stream writer...</li> <li><a href="https://github.com/apache/beam/commit/b2b2f1417ed905b76c915c7b7aea757e1cd6f0a7";><code>b2b2f14</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/33486";>#33486</a>: Cherry-pick <a href="https://redirect.github.com/apache/beam/issues/33483";>#33483</a> to release 2.62.0 (Fix bounded ...</li> <li><a href="https://github.com/apache/beam/commit/88f1f3c0f554cdfde31b4cd1403d6db8f0034526";><code>88f1f3c</code></a> Merge pull request <a href="https://redirect.github.com/apache/beam/issues/33479";>#33479</a>: [Release-2.62] cherry-pick <a href="https://redirect.github.com/apache/beam/issues/33472";>#33472</a> into release bra...</li> <li><a href="https://github.com/apache/beam/commit/e8cd8a6c826a831cb8c88b8fe0d1af9efc0e39df";><code>e8cd8a6</code></a> Fix bounded trie merge when merging on empty trie</li> <li><a href="https://github.com/apache/beam/commit/e72ad94753cacff6cfbb136eed60aaaed0c9166e";><code>e72ad94</code></a> Make Beam expansion service shadowJar multi-release so it processes multi-rel...</li> <li>Additional commits viewable in <a href="https://github.com/apache/beam/compare/v2.61.0...v2.62.0";>compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
